VFX – Dawn of the digital era

Ramji digs into the development of motion capture, and the effect it has had on the creative freedom for filmmakers of modern cinema

In the first two parts, we saw how special effects involved mostly in-camera techniques, and CGI (Computer Generated Imagery) started to enter from early 90s. Nineties was the cusp when in-camera effects slowly started to give way to digital effects. Several pathbreaking films used digital technology effectively in creating complete awesomeness. The digital effects domain was nascent, and several techniques or software needed to be invented just to achieve a specific requirement. This opened up several job opportunities.

Capture your movement

Motion capture systems have been in the research since 1970s and 80s were also introduced to the films to create animations that are hyper realistic. The concept was invented by Max Fleischer in 1915 where animators used a projection system that projected movements of actors on a screen. The artists would trace every frame of the actor that made the animated character move realistically. This process was called rotoscoping.

Modern motion capture systems involve human actors wearing a special suit fitted with markers that react with infrared light. A special camera is used to capture the actor’s movements. The camera only captures the actor’s movements through the infrared light information and sends this data to the animation software. This movement data is then applied to the digital 3D character and voila! The digital character moves just like the actor!

Moonwalking Skeleton

Michael Jackson made it popular through his famous skeleton dance in Ghost (1996). His dance moves were motion captured and those movements were applied to the skeleton that dances just like the King of Pop himself.

Slicing through time

The Matrix (1999) used several VFX shots to create extraordinary visuals. It involved several green screen shots where the backgrounds both digital and real were combined.

The Matrix also used a new technique called Time Slice where the actors freeze in an action scene and the camera goes around them in a full circle. Once that is complete, the action sequence continues. This was achieved by placing several still cameras (the black dots in the middle of the green screen) connected to a common firing system. Movie cameras were placed at each end of this circuit that would capture the live actions. In the middle of the action sequence, the still cameras all fire either at the same time or with a slight delay to each other depending on the scene requirement. These still images are processed through a software system that interpolates the frames in between each still image and create that seamless time slice. The green screen will be replaced with a digital background. The Matrix had several such sequences that made us watch like this. Truly a jaw-dropping experience!

Extend that set!

The usage of CGI slowly increased in films to create both visible and invisible effects. Invisible effects included digital set extensions. This is a technique in which a virtual 3D model of the set is built on top of the real set, and it is matched with the camera view seamlessly. Gladiator (2000) used it to digitally extend the sets of the Roman Colosseum. The actual set was only one or two storeys high, and the rest of the colosseum was CGI. This helped the movie-makers film at real locations to create authentic visuals and this provided enough references for the digital artists to extend the set virtually.

Digital soldiers

In The Lords of Rings trilogy (2001 – 2003) along with Harry Potter series (2001 onwards) involved several VFX techniques to bring the magic in the novels/script to the screen. Not to mention these films were made possible only with the help of modern VFX techniques. The war scenes in The Lord of the Rings had digital warriors fighting each other. An AI (Artificial Intelligence) based software called Massive was used to generate thousands of these digital soldiers. The AI system built in Massive was used to differentiate between a soldier and his enemy so that each digital soldier always attacks only the enemy. How did this work? The software has a library of fighting actions captured through motion capture systems, and these actions were assigned randomly or in response to the opponent soldier at run time. This made the digital soldiers fight look realistic.

Since then, it has been used extensively to generate huge crowds in films. This solved a lot of logistical problems for the film makers.

Another technique to populate crowds these days. A small group of people are shot with various random actions from several angles, and they are duplicated digitally and placed in various parts of the set to show huge crowd. This small group is mixed with 3D models of people doing different actions and they are filled in the stadium. Look at this example of crowd simulation in a recent TV show.

Performance Capture

A new technique was developed for Avatar (2009) and was called Performance Capture. In this technique, in addition to the motion capture suit and sensors, a camera is attached to the actor’s head, focusing on the actor’s face. The face has additional tiny markers that move as the actors emote. In Avatar, this technique was used to transpose the expressions to the Na’Vi characters which were digitally created.

Gradually the components of VFX kept increasing as years passed by and it reached an extent where only the actors were real and the complete set or the background was digitally generated. Look at this behind the scenes (BTS) shot from the famous series Game of Thrones (2011 onwards).

Virtual Production

This field is constantly evolving, and the latest trend is a throwback to the olden techniques using modern technology. Yes, the back projection (a technique to project a moving background while the actors rode on stationary vehicles) is back in new form. Virtual Production is a new technique used in film making. The back projection had a problem when the camera angle dynamically changes, and the resultant image appears fake. This is due to the parallax effect. When a camera moves around an object, the foreground and the background appear to move at different speeds. In a back projection technique, this could not be achieved because the background is a flat projection. Now, virtual production technique involves a semi-circular digital screen on which the required backgrounds for the scene are projected while actors perform in front of the screen.

The virtual backgrounds are created separately and displayed on this digital screen. The real camera is synchronized with the virtual camera in the background. So, when the real camera moves, the virtual camera mimics the movements of the real camera and the perspective of the virtual background changes to match the angle of the real camera. This technique allows the filmmakers to get the maximum out of the camera and allows actors to react appropriately. The background also reflects on the foreground actors or objects thereby giving it a very realistic look.

Sify Technologies – Digital Services

Not only virtual backgrounds, but even actors are also virtually created. Unreal Engine’s Metahuman and Unity3D’s digital humans are experimenting in creating life-like humans. A hyper-realistic film with digital environments and characters is indeed a possibility.

Modern VFX techniques

With the advent of digital technology, special effects and visual effects have grown leaps and bounds. VFX gives the filmmakers the chance to explore a variety of themes and stories to enthrall the audience. The in-camera effects were even more difficult and challenging to filmmakers but as technologies evolved, control over different elements is easily achieved using modern VFX techniques. So, the next time you go and watch a movie, remember that so many techniques are used to make you believe and experience the story.

In case you missed:

None Found