Motion capture, motion tracking, or mocap is a technique of digitally recording movements for entertainment, sports, and medical applications.
Methods and Systems
Motion tracking or motion capture started as a photogrametric analysis tool in biomechanics research in the 1970s and 1980s,
and expanded into education, training, sports and recently computer animation for
cinema and video games as the technology matured. A performer
wears markers near each joint to identify the motion by the positions or angles between the markers. Acoustic, inertial,
LED, magnetic or reflective markers, or combinations of any of these, are tracked,
optimally at least two times the rate of the desired motion, to submillimeter positions. The motion capture computer software
records the positions, angles, velocities, accelerations and impulses,
providing an accurate digital representation of the motion.
In entertainment applications this can reduce the costs of animation which otherwise requires the animator to draw each frame,
or with more sophisticated software, key frames which are interpolated by the software. Motion capture saves time and
creates more natural movements than manual animation, but is limited to motions that are anatomically possible. Some applications
might require additional impossible movements like animated super hero martial arts or stretching and squishing that are
not possible with real actors.
In biomechanics, sports and training, real time data can provide the necessary information to diagnose problems or suggest
ways to improve performance, requiring motion capture technology to capture motions up to 140 miles per hour for a golf
swing.
Optical systems
Optical systems triangulate the 3D position of a marker between one or more
cameras calibrated to provide overlapping projections. Tracking a large number of markers or multiple performers or expanding the
capture area is accomplished by the addition of more cameras. These systems produce data with 3 degrees of freedom for each
marker, and rotational information must be inferred from the relative orientation of three or more markers; for instance
shoulder, elbow and wrist markers providing the angle of the elbow...
Optical: Passive Markers
A dancer wearing a suit used in an optical motion capture system
Passive optical system use markers coated with a Retroreflective material to
reflect light back that is generated near the cameras lens. The cameras sensitivity can be adjusted taking advantage of most
cameras narrow range of sensitivity to light so only the bright markers will be sampled ignoring skin and fabric.
The centroid of the marker is estimated as a position within the 2 dimensional image that is captured. The grayscale value of
each pixel can be used to provide sub-pixel accuracy.
An object with markers attached at known positions is used to calibrate the cameras and obtain their positions and the lens
distortion of each camera is measured. Providing two calibrated cameras see a marker, a 3 dimensional fix can be obtained.
Typically a system will consist of around 6 to 24 cameras. Systems of over two hundred cameras exist. Extra cameras are required
for full coverage around the capture subject. Typical eight camera systems are about $150,000 US.
Professional vendors have sophisticated constraint software to reduce problems from marker swapping since all markers appear
identical. Unlike active marker systems and magnetic systems, passive systems do not require the user to wear wires or electronic
equipment. The markers are usually attached directly to the skin (as in biomechanics), or they are velcroed to a performer
wearing a full body spandex/lycra suit designed specifically for motion capture. This type of system can capture large numbers of
markers at frame rates as high as 2000fps. The frame rate for a given system is often traded off between resolution and speed so
a 4 megapixel system runs at 370 hertz normally but can reduce the resolution to .3 megapixels and then run at 2000 hertz.
Optical: Active marker
Active optical systems triangulate positions by illuminating one LED at a time very quickly or multiple LEDs but sophisticated
software to identify them by their relative positions, somewhat akin to celestial navigation. Rather than reflecting light back
that is generated externally, the markers themselves are powered to emit their own light. Since Inverse Square law provides 1/4
the power at 2 times the distance, this can increase the distances and volume for capture. ILM used active Markers in Van Helsing
to allow capture of the Harpies on very large sets. The power to each marker can be provided sequentially in phase with the
capture system providing a unique identification of each marker for a given capture frame at a cost to the resultant frame rate.
The ability to identify each marker in this manner is useful in realtime applications. The alternative method of identifying
markers is to do it algorithmically requiring extra processing of the data.
Optical: Time modulated active marker
A high-resolution active marker system with 3,600 × 3,600 resolution at 480 hertz providing real time submillimeter
positions.
Active marker systems can further be refined by strobing one marker on at a time, or tracking multiple markers over time and
modulating the amplitude or pulse width to provide marker ID. 12 megapixel spatial resolution modulated systems show more subtle
movements than 4 megapixel optical systems by having both higher spatial and temporal resolution. Directors can see the actors
performance in real time, and watch the results on the mocap driven CG character. The unique marker IDs reduce the turnaround, by
eliminating marker swapping and providing much cleaner data than other technologies. LEDs with onboard processing and a radio
synchronization allow motion capture outdoors in direct sunlight, while capturing at 480 frames per second due to a high speed
electronic shutter. Computer processing of modulated IDs allows less hand cleanup or filtered results for lower operational
costs. This higher accuracy and resolution requires more processing than passive technologies, but the additional processing is
done at the camera to improve resolution via a subpixel or centroid processing, providing both high resolution and high speed.
These motion capture systems are typically under $50,000 for an eight camera, 12 megapixel spatial resolution 480 hertz system
with one actor.
Semi-passive imperceptible markers can self-compute their location when lit by mobile multi-LED emitters, e.g. in a moving car.
With Id per marker, these tags can be worn under clothing and tracked at 500 Hz in broad daylight.
Optical: Semi-passive Imperceptible Marker
One can reverse the traditional approach based on high speed cameras. Systems such as Prakash use inexpensive multi-LED high
speed projectors. The specially built multi-LED IR projectors optically encode the space. Instead of retro-reflective or active
light emitting diode (LED) markers, the system uses photosensitive marker tags to decode the optical signals. By attaching tags
with photo sensors to scene points, the tags can compute not only their own locations of each point, but also their own
orientation, incident illumination, and reflectance.
These tracking tags that work in natural lighting conditions and can be imperceptibly embedded in attire or other objects. The
system supports an unlimited number of tags in a scene, with each tag uniquely identified to eliminate marker reacquisition
issues. Since the system eliminates a high speed camera and the corresponding high-speed image stream, it requires significantly
lower data bandwidth. The tags also provide incident illumination data which can be used to match scene lighting when inserting
synthetic elements. The technique is therefore ideal for on-set motion capture or real-time broadcasting of virtual sets.
Non-Optical Systems
Inertial systems
Inertial Motion Capture technology is based on miniature inertial sensors, biomechanical models and sensor fusion algorithms.
It's an easy to use and cost-efficient way for full-body human motion capture. The motion data of the inertial sensors
(Inertial_guidance_system) is transmitted wirelessly to a PC or laptop, where
the full body motion is recorded or viewed. No external cameras, emitters or markers are needed for relative motions. Inertial
mocap systems capture the full 6 degrees of freedom body motion of a human in real-time. Benefits of using Inertial systems
include; No solving, freedom from studio's as most systems are portable, and large capture areas. These systems are similar to
the WII controllers but much more sensitive and having much greater resolution and update rate. They can accurately measure the
direction to the ground to within a degree. Base suits tend to be in the $50,000 range, however the cost of a system often
relates to the quality of data it produces.
Mechanical motion
Mechanical motion capture systems directly track body joint angles and are often referred to as exo-skeleton motion capture
systems, due to the way the sensors are attached to the body. A performer attaches the skeletal-like structure to their body and
as they move so do the articulated mechanical parts, measuring the performer’s relative motion. Mechanical motion capture systems
are real-time, relatively low-cost, free-of-occlusion, and wireless (untethered) systems that have unlimited capture volume.
Typically, they are rigid structures of jointed, straight metal or plastic rods linked together with potentiometers that
articulate at the joints of the body. These suits tend to be in the $25,000 to $75,000 range plus an external absolution
positioning system.
Magnetic systems
Magnetic systems calculate position and orientation by the relative magnetic flux of three orthogonal coils on both the
transmitter and each receiver. The relative intensity of the voltage or current of the three coils allows these systems to
calculate both range and orientation by meticulously mapping the tracking volume. Since the sensor output is 6DOF, useful results can be obtained with two-thirds the number of markers required in optical
systems; one on upper arm and one on lower arm for elbow position and angle. The markers are not occluded by nonmetallic objects
but are susceptible to magnetic and electrical interference from metal objects in the environment, like rebar (steel reinforcing
bars in concrete) or wiring, which affect the magnetic field, and electrical sources such as monitors, lights, cables and
computers. The sensor response is nonlinear, especially toward edges of the capture area. The wiring from the sensors tends to
preclude extreme performance movements. The capture volumes for magnetic systems are dramatically smaller than they are for
optical systems. With the magnetic systems, there is a distinction between “AC” and “DC” systems: one uses square pulses, the
other uses sine wave pulse.
The procedure
In the motion capture session, the movements of one or more actors are sampled many times per second. High resolution optical
motion capture systems can be used to sample body, facial and finger movement at the same time.
A motion capture session records only the movements of the actor, not his visual appearance. These movements are recorded as
animation data which are mapped to a 3D model (human, giant robot, etc.) created by a computer artist, to move the model
the same way. This is comparable to the older technique of rotoscope where the visual
appearance of the motion of an actor was filmed, then the film used as a guide for the frame by frame motion of a hand-drawn
animated character.
If desired, a camera can pan, tilt, or dolly around the stage while the actor is performing and the motion capture system can
capture the camera and props as well. This allows the computer generated characters, images and sets, to have the same
perspective as the video images from the camera. A computer processes the data and displays the movements of the actor, as
inferred from the 3D position of each marker. If desired, a virtual or real camera can be tracked as well, providing the desired
camera positions in terms of objects in the set.
A related technique match moving can derive 3D camera movement from a single 2D image
sequence without the use of photogrammetry, but is often ambiguous below centimeter resolution, due to the inability to
distinguish pose and scale characteristics from a single vantage point. One might extrapolate that future technology might
include full-frame imaging from many camera angles to record the exact position of every part of the actor’s body, clothing, and
hair for the entire duration of the session, resulting in a higher resolution of detail than is possible today.
After processing, the software exports animation data, which computer animators can associate with a 3D model and then
manipulate using normal computer animation software. If the actor’s performance was good and the software processing was
accurate, this manipulation is limited to placing the actor in the scene that the animator has created and controlling the 3D
model’s interaction with objects.
Advantages
Mo cap offers several advantages over traditional computer animation of a 3D
model:
- More rapid, sometimes even real time results can be obtained.
- The amount of work does not vary with the complexity or length of the performance to the same degree when using traditional
techniques.
- Complex movement and realistic physical interactions such as secondary animation, weight and exchange of forces can be more
easily recreated in a physically accurate manner.
- Mocap technology allows one actor to play multiple roles within a single film.
Advantages over live action
In movies that contains CGI in such large amounts that the actors would have to stay in front of a bluescreen and interact
with invisible computer animated characters which is added later, trying to fit into a computer animated world, it is sometimes
less problematic to make everything digital, including the actors. This way, all elements would fit together naturally and have
the same visual look.
- The director can choose any angle he or she desires from a scene, including angles that would have been hard or impossible in
a live action movie.
- Limitless possibilities for rotating effect.
- Costumes, make-up, body size and age can be changed to whatever is needed.
- The characters will blend perfectly in with their digital environments.
- There is no need to have light, colors and filters in mind when filming the motions, as this will be added digitally
later.
Disadvantages
- Specific hardware and special programs are required to obtain and process the data.
- The cost of the software and equipment, personnel required can be prohibitive for small productions.
- The capture system may have specific requirements for the space it is operated in.
- When problems occur it is sometime easier to reshoot the scene rather than trying to manipulate the data. Only a few systems
allow real time viewing of the data to decide if the take needs to be redone.
- Applying motion to quadruped characters can be difficult.
- The technology can become obsolete every few years as better software and techniques are invented.
- The results are limited to what can be performed within the capture volume without extra editing of the data.
- Movement that does not follow the laws of physics generally cannot be represented.
- Traditional animation techniques such as added emphasis on anticipation and follow through, secondary motion or manipulating
the shape of the character as with squash and stretch animation techniques are generally not applicable.
- If the computer model has different proportions from the capture subject artifacts may occur. For example, if a cartoon
character has large, over-sized hands, these may intersect strangely with any other body part when the human actor brings them
too close to his body.
- The real life performance may not translate on to the computer model as expected.
Applications
Some video games use motion capture to animate athletes, martial artists, and other
in-game characters.
Movies use motion capture for CG effects, in some cases replacing traditional cell animation, and for completely
computer-generated creatures, such as Gollum,
The Mummy, and King Kong.
was the first movie made primarily with motion
capture, although many character animators also worked on the film.
In producing entire feature films with Computer animation, the industry is
currently split between studios that use Motion Capture, and studios that do not. Out of the three nominees for the 2006
Academy Award for Best Animated Feature, two of the nominees
("Monster House" and the winner "Happy Feet") used Motion Capture, and only Pixar's Cars was
animated without Motion Capture. In the ending credits of Pixar's latest film "Ratatouille", a stamp appears labelling the film as "100% Pure Animation -- No Motion Capture!" On
the other hand, Pixar's parent The Walt Disney
Company has announced that it will distribute Robert Zemeckis's A
Christmas Carol to be produced using "Performance Capture," the motion-capture approach that Zemeckis first used on
The Polar Express.
Virtual Reality and Augmented Reality
allow users to interact with digital content in real-time. This can be useful for training simulations, visual perception tests,
or performing a virtual walk-through in a 3D environment.
Gait analysis is the major application of motion capture in clinical medicine.
Motion capture technology is frequently used in digital puppetry systems to aid in
the performance of computer generated characters in real-time.
Related techniques
Facial motion capture is utilized to record the complex movements in a human
face, especially while speaking with emotion. This is generally performed with an optical setup using multiple cameras arranged
in a hemisphere at close range, with small markers glued or taped to the actor’s face.
Inertial systems use devices such as accelerometers or gyroscopes to measure positions and angles. They are often used in
conjunction with other systems to provide updates and global reference, since they only measure relative changes, not absolute
position.
RF (radio frequency) positioning systems are becoming more viable as higher frequency RF devices allow greater precision than
older RF technologies. The speed of light is 30 centimeters per nanosecond (billionth of a second), so a 10 gigahertz (billion
cycles per second) RF signal enables an accuracy of about 3 centimeters. By measuring amplitude to a quarter wavelength, it is
possible to improve the resolution down to about 8 mm. To achieve the resolution of optical systems, frequencies of 50 gigahertz
or higher are needed, which are almost as line of sight and as easy to block as optical systems. Multipath and reradiation of the
signal are likely to cause additional problems, but these technologies will be ideal for tracking larger volumes with reasonable
accuracy, since the required resolution at 100 meter distances isn’t likely to be as high.
An alternative approach was developed where the actor is given an unlimited walking area through the use of a rotating sphere,
similar to a hamster ball, which contains internal sensors
recording the angular movements, removing the need for external cameras and other equipment. Even though this technology could
potentially lead to much lower costs for mocap, the basic sphere is only capable of recording a single continuous direction.
Additional sensors worn on the person would be needed to record anything more.
A studio in the Netherlands is using a 6DOF (Degrees of freedom) motion platform with an integrated omni-directional treadmill
with high resolution optical motion capture to achieve the same effect. The captured person can walk in an unlimited area,
negotiating different uneven terrains. Applications include medical rehabilitation for balance training, biomechanical research
and virtual reality.
Several research groups around the world tackle the task of Markerless Motion Capture, e.g.
Programs / Systems
- Animazoo IGS Vs 8.3: IGS 8.3 software is
the latest software to be used with Animazoo motion capture products. IGS 8.3 has been develop over 10 years of working with
Games Developers, Animation Studio's, and Biomechanics experts.
- JZZ Technologies, Inc: 3D
non-camera based motion capture system utilizing the E-Factor 3D Motion Capture Engine
- Vicon
BLADE: Scalable real-time and systems killer kinematic solving and retargeting full performance motion capture for hands,
body and face.
- Phoenix Technologies Inc: 3D Real-Time
Active Optical Motion Capture Systems with advanced patented wide angle active marker tracking technology, innovative accessories
and software for full body motion capture including facial mocap.
- Moven
Studio: a software application for motion capturing with the Moven suit. Works with Microsoft Windows XP and Windows Vista.
- Organic Motion: By Organic Motion INC, real-time markerless motion capture. Doesn't require a body suit, markers.
- Imocap: By ILM, a proprietary on-set computer-vision based
full-body motion capture system first used in Pirates of the
Caribbean: Dead Man's Chest. The actors wear checkered bands.
- Mova Contour reality capture: Trackable facial
and surface motion capture using phosphorescent makeup applied to skin or dyed into fabric. An array of cameras alternates
imaging the phosphor pattern for 3D geometry and natural skin/cloth color for textures.
- Image Metrics: Proprietary facial
motion capture service, no makeup, no spandex suit, no special hardware.
- PhaseSpace Recap: Motion capture editor,
no makeup, black spandex suit, Modulated LED hardware.
- Pure Data + Pidip: motion capture, open source program
- Eyesweb: motion
capture, freeware
- Measurand Inc: Portable, durable,
wireless, and real-time motion capture
- NaturalPoint Arena: Full body
motion capture solution which generates real time skeleton data. Used with OptiTrack cameras to preprocess images for minimal
processor hit.
- HumanEVA database:
Synchronized Video and Motion Capture Dataset for Evaluation of Articulated Human Motion
- Prakash :
Lighting-Aware Motion Capture Using Imperceptible Photosensing Markers and Multiplexed Illuminators
- I-CubeX: A sensor toolkit specifically designed
to capture all kinds of human motions using a variety of technologies
In fiction
- In the 2001 television film How to Make a Monster, a motion
capture suit is brought to life and becomes a monster as a result of the lightning strike on the AI chip, and starts hunting the
programmers in a deadly game.
- In the Mortal Kombat series, starting with 2002 game Mortal Kombat: Deadly Alliance, a character called Mokap was integrated to the roster of fighters. As his name implies, he was a motion capture actor in some of
Johnny Cage's films.
See also
External links
This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer)
