It is said AI in filmmaking will change cinema. Satyen K. Bordoloi argues that the opposite is also true as he outlines the amazing tools that deals like this could unleash into the world.
In the 2002 film Awara Paagal Deewana, Akshay Kumar copies the lobby fight sequence from The Matrix. It was achieved by flying in the team that created the original to Mumbai. Naturally, it cost a bomb. Today, AI can achieve better quality at a fraction of the cost. A deal to make something like this possible, transforming both AI and filmmaking, was signed between Lionsgate and Runway.
Lionsgate owns some of the most successful film franchises, including John Wick, The Hunger Games, Rambo, Saw, Blair Witch, The Expendables, and The Twilight Saga. Runway is a pioneer in generative AI, and is one of the first companies with an excellent text-to-image, then text-to-video model.
As expected, this deal has the world of both cinema and AI buzzing. That it will change the world of filmmaking is being acknowledged. What isn’t is how it’ll also transform the landscape of AI.
WHAT’S IN IT FOR RUNWAY:
One of the big global fights is between companies who create content and AI companies who need them to train their models. Media companies across the world, particularly in the US, including news and novel publications, are up in arms against LLM model makers, claiming – rightly so – that their copyright has been infringed by these AI companies using their proprietary content to train their models. The world of AI has so far been the wild wild west where anything went. But as creators are reclaiming the rights to their work, lawsuits are flying thick and fast.
At the most basic level, this deal gives Runway some of the best, most cinematic content ever produced to train their models legitimately. At a deeper level, this deal also gives both Runway and the text-to-video industry a legitimacy in the film business they have not yet had, coming as it does a year after the SAG-AFTRA strike to limit the use of AI by production companies. Deals between other AI and production companies might already be on the table, portending exciting times for both AI and cinema.
WHAT’S IN IT FOR LIONSGATE:
It has barely been a year and a half since the first video created by AI was posted online. The clip of Will Smith eating spaghetti has noticeable glitches and awkward movements. The facial expressions are exaggerated and weird, lacking the smooth realism of actual footage. But in that time, AI video generation has leapfrogged multiple generations. It took humans six and a half decades to go from the first flight to landing on the moon. We have gone from a glitchy, surreal video to being able to make entire films with just AI in under two years. The AI bait was dangling for the first bite, which happened to be Lionsgate.
This gives Lionsgate the first-mover advantage. Other studios in Hollywood are already making their own AI systems, most notably Disney. This deal is different because this is a first-of-its-kind collaboration between a movie-making company and an actual AI company. Even Netflix is primarily a tech company, then a distribution platform, and a production house last. This marriage between two masters in their domains is certain to birth revolutionary offspring.
UNIQUE PRODUCTS:
Giving Runway Lionsgate’s entire library to play with could lead to some fascinating products which we can only imagine right now. The lowest hanging fruits would be in how films are made. Say you are a studio in Mumbai wanting to make a film similar in look and style to John Wick, except with Akshay Kumar playing the lead. Instead of cutting and pasting frames and shots from John Wick to act as reference for your production, you can ask LionsWay (Runway-Lionsgate AI system) and it’ll not only give you the specific references you want from the film but make new ones in the same style based on your needs. Your reference library – something that every Bollywood production makes – can be made more exact and in hours, not weeks. What’s more, instead of Keanu Reeves’ face, the AI will give it to you with Akshay Kumar’s face.
Secondly, some of the establishing shots can be created by LionsWay, saving a lot of money. Having the shots before you go on the floor also means you can match the lighting. Maybe Lionsgate can even create lighting rigs connected to the AI to give the lighting a cinematographer needs at the push of a button.
The biggest advantage I foresee with such a system is that instead of a simple storyboard or previsualization, you could make the whole movie with AI before it is made. The writer, director, cinematographer, and editor can sit with LionsWay to make all the shots from the screenplay and make an AI version of the film before the actual film is shot. Currently, the storyboards of many filmmakers are detailed with images, sketches and videos from different references. Such or other tools could give them a way to make the entire film before making the film. What is amazing is that part of this AI film could end up in the final film as well.
The actual production could thus be a question of just copying from the pre-made film. The amount of clarity this will give a team is immense, not to mention the time and money saved on making the film. Editing and post-production tools, including VFX, will get an AI twist with a deal like this, where voice and text commands could be used to create lots of effects. An entire toolset of music created from Lionsgate’s existing library and fed into LionsWay could help create reference music to fit the mood of the scenes in the film, without needing to copy reference music from other films as is done currently.
More fantastical things could be in the offing way into the future. This LionsWay could be a platform like Netflix, where for a small price, a user like you and me can pay to have our face replaced over Keanu Reeves’ in John Wick as we derive unimaginable pleasure watching ourselves bashing villains to pulp. LionsWay could also be a creation tool for social media content creators. Rather than leasing out footage from old films, such an app could license new content created from their old content. This is an entirely new business model which has never been heard of before.
Then there’s the business of fan films. Like fan fiction, you could use elements from a film you like to create your own short film or feature. Say, you want to use the Continental Hotel in the John Wick franchise to tell a new story of not John, but say his daughter – Joan Wick, you could create that short film and maybe even host it on LionsWay.
Licensing out APIs to others like Adobe via a B2B model could be in the works as well.
HOW THIS DEAL WILL ALTER THE AI LANDSCAPE:
In two years, generative AI has not only changed the paradigm of many industries but made NVIDIA a $3 trillion company. That’s because LLMs need a lot of GPUs to power the computation required for their responses. Can you imagine how much more this computation will shoot up when text-to-video tools become ubiquitous?
A typical query to a large language model might use a fraction of a GPU’s capacity, measured in milliseconds of GPU time. However, creating even one second of video using text-to-video is exponentially more computationally intensive. For every second, the models need to generate at least 24 frames for a smooth video, using exponentially more complex computations than those used in text-to-text or text-to-image models.
As the use of text-to-video or image-to-video AI rises among the general populace, thanks to deals like this one, the need for GPUs will shoot up as an exponent, giving a shot in the arm to the entire AI industry. The innovations required to make text-to-video possible will also dramatically transform the AI field, not to mention the server farms and energy generation that will be required to run and cool these facilities.
Akshay Kumar can fulfil his fetish to play Neo, John Wick, Rambo, or any other Hollywood hero he likes, without flexing any of his well-exercised muscles, by simply using just AI. Considering how much of a film’s budget he hogs, this is a blessing that this Awara Paagal Deewana wishes he had years ago.
In case you missed:
- What are Text-to-Video Models in AI and How They are Changing the World
- How AI video creators like OpenAI’s Sora will change cinema forever
- ‘Now and Then’: Time Travelling with The Beatles’ Last AI Song
- Data Scraping AI Companies & Writers Fight to Define Future of AI
- 10 Years of ‘Her’: What the Film Got Right about AI, What It Did Not
- Copy Of A Copy: Content Generated By AI, Threat To AI Itself
- PSEUDO AI: Hilarious Ways Humans Pretend to be AI to Fool You
- AI vs. Metaverse & Crypto: Has AI hype lived up to expectations
- OpenAI’s Secret Project Strawberry Points to Last AI Hurdle: Reasoning
- Susan Wojcicki: The Screaming Legacy of The Quiet Architect of the Digital Age
