First generative AI like Dall-E, Stable Diffusion and MidJourney, then ChatGPT and clones have woken us to the potential of AI but its full possibility will surface in the first truly zero-budget film made without a camera writes Satyen K. Bordoloi.
Democratization of cinema is a term used often to describe the digitization of the means of filmmaking. Everyone who has a mobile, it is said, can become a filmmaker. But anyone who has tried to shoot anything decent with a mobile knows how tough this is. And even if visuals are cinematic, how do you record flawless sound?
This is about to see a dramatic transformation with the advent of Artificial Intelligence. The future is so strange, it defies imagination. But the best thing: we will finally achieve democratic filmmaking where everyone will have the power to make cinematic films at home: even those who do not own any camera whatsoever.
Legoization is ‘Unreal’
Over the last forty years, a good part of the developments in television, cinema and healthcare imaging – both hardware and software – have come from the gaming industry. Much of artificial intelligence now is run on chips made for gaming. But nowhere are the effects of gaming more visible than today where the visual quality of games has reached cinematic levels thanks to advancements in both hardware and software. You can no longer tell if you are playing a game or watching a Hollywood blockbuster. At over $350 billion, per year global gaming industry revenues are over five times that of films.
It was hence expected that the world of games would invade filmmaking. They have and the way they have done this is just ‘unreal’ – via Unreal Engine – the ‘Lego’ tool that has totally revolutionized filmmaking in just the last few years.
Back in 2000, Matrix Reloaded encountered a problem. Neo fights a clone army of Agent Smiths. Wachowski’s vision could not be achieved the normal way. So, they created a virtual film set inside a computer where they could shoot in ways impossible in the real world. Mixing this with live-action footage created what was one of the most mind-boggling action sequences of its time. Looking back at the scene, you can differentiate the real and virtual footage from the weird camera movements and the lack of expression on the digital characters’ faces.
That virtualization in Matrix Reloaded, which took millions of dollars, has been brought alive for everyone by Epic Games via Unreal Engine and most of its tools are free to use. As the name suggests, what you can do in it is just unreal.
Filmmaking in general as well, is like making and solving your own jigsaw puzzle. Little bits of hundreds or thousands of shots are put together to make a coherent story. But what Unreal Engine does is the absolute Legoization of cinema. You can pick elements to create visuals on the frame like making Lego structures.
You need an outdoor set? Pick the mountains you want in your background. Pick boulders and set them in the foreground. Move the sun and thus the light you want – from east to west, the other way around or even something bizarre and within minutes you have your set ready. Place a digital character inside this, or a real character before a huge LED screen displaying this set, and voila, you have your production ready in minutes not days, weeks or months. You have a city to build? Just pick the type of buildings you want and place them where you want.
Unreal Engine has been used to make short films entirely inside computers. They have also become an integral part of filmmaking eliminating the need for a greenscreen altogether. The film 300 was shot entirely inside a studio in front of a greenscreen but took a lot of hard work, planning and cost to get right. Today, 300 could have been made at a fraction of the cost and with better visual effects using Unreal Engine, as The Mandalorian has been.
As you have guessed, this type of filmmaking is magical but still takes some time, effort and money. But what is coming next, is beyond truly ‘insane’ and beyond imagination.
The Future of Filmmaking is AI
In September and October 2022, both Meta and Google announced the launch of their individual text prompt to video makers. Back then it was dubbed the Dall-E for video – Dall-E being the first text prompt to image maker released a few months prior. Both the companies and almost every other AI company worth their name from Microsoft to the flavour of the season OpenAI, are also working on their own prompt to video makers.
It is just a matter of time when, like you have seen the implosion of images created by AI, you will have both bizarre and beautiful videos created by AI flooding your timeline on all platforms. And within months, its quality will be so good, you will wonder if it has been shot by a master director of photography – the same feeling you get when you see images created by AI right now.
It will begin slow and stupid with people trying to make strange things in the hopes that it goes viral. But once that novelty wears off, and the technology gets better, you will have videos the likes of which do not exist today. Like prompt-image making has given wings to people’s creativity, prompt video-making will expand the imagination of the world beyond anything we have seen so far – surpassing even what is being done with images.
But what about sound, you will ask. The voice barrier has already been breached. There are already hundreds of apps, software and websites where with a few dollars, you can literally clone anyone’s voice. This has led to a new version of phone scams where family members hear this AI-generated voice of their loved ones begging for help, to transfer money to scamsters. We wrote about this in Sify a while ago.
With everything – images, sound and video created by AI – you have the perfect recipe for zero-budget filmmaking. Within the next two to five years, the following is how many will make films.
How to make a Truly Zero Budget Film
Gayatri wakes up one day with the memory of a strange dream and decides to make a film. The sci-fi idea is about a time travelling triple agent who passes information back and forth in time to topple dictatorships with the end reveal that the spy is not human but an alien from another dimension who is also passing information between dimensions to truly weird-looking creatures. A truly tough film to make in every sense. Especially for Gayatri who is a machine operator at a radiology lab. But she is bored with her work and life and craves a challenge. So, she decides to use available and free AI to make her film.
The first step: write a screenplay. She spends time with the ChatGPT 7 plugin on Final Draft screenwriting software, both together having become advanced enough to write good screenplays on its own with word prompts. It takes her six days to send good prompts ChatGPT’s way so it can write the screenplay on Final Draft. On the seventh day, she is ready with a 100-page screenplay she titles ‘Triple Dimension Agent’. Final Draft has another plugin that extracts each character in the screenplay and checks out who in the history of the world would be best suited to play each part.
Since this is an interdimensional adventure, you want at least an international cast. But since Gayatri does not have a budget, she goes for people whose rights are available free on open source. Dilip Kumar from Madhumati has fallen to open source. So, she tells her AI system to extract Dilip Kumar, colourise him and also synthesize his voice. Your lead, the interdimensional triple agent is ready.
For the female lead, she extracts Greta Garbo from Ninotchka and the voice of Edith Piaf from her French films of the 1940s and 50s. With some changes to some stock creatures available on Unreal Engine, casting is complete.
Then begins pre-production in Unreal Engine. Gayatri takes a few more days – with the help of Final Draft itself – to find the perfect location in the universe of Unreal Engine and tweak it to her needs though the AI has done 80% of her job. Gayatri does this scene by scene and it takes her a week as there are 120 scenes in the film including montages and 25 key locations.
Once this is done, she goes for production. She places Dilip Kumar and Garbo in the opening shot inspired by Casablanca but set in a futuristic bar where the Pianist Sam is a sloth bear-looking alien. She sets the movement of the main actors and sets the movement for the digital extras in the scene using nothing but voice prompts to the AI system. The dialogues are extracted automatically and Gayatri puts in the intonations she wants the characters to portray.
She goes scene by scene and realises that filmmaking is tough for she has managed to shoot only 5 minutes’ worth of footage in the entire day. So, she speeds up and has the film ready in the next 25 days, including a bit of post-production and background music which is taken care of by the AI itself.
This is the future of filmmaking. But there is another dimension that will emerge: personal filmmaking i.e. films starring you. Filmmaker Joe Russo described this perfectly when he said: “You could walk into your house and tell the A.I. on your streaming platform. ‘Hey, I want a movie starring my photoreal avatar and Marilyn Monroe’s photoreal avatar. I want it to be a rom com because I have had a rough day,’ and it renders a very competent story with dialogue that mimics your voice. It mimics your voice, and suddenly now you have a rom com starring you that is 90 minutes long. So you can curate your story specifically to you.”
Yogi Berra once said: “The future ain’t what it used to be.” AI has brought us to a time when that is literally true. We could once predict the future with a bit of accuracy. But now the future is an uncharted road filled with all manner of strange things. One of the strangest are films made by people on their laptops without any other equipment.
In case you missed:
- How AI video creators like OpenAI’s Sora will change cinema forever
- How Lionsgate-Runway Deal Will Transform Both Cinema & AI
- What are Text-to-Video Models in AI and How They are Changing the World
- ‘Now and Then’: Time Travelling with The Beatles’ Last AI Song
- Personal AI’s Future Is Here: Not What You Expected; Close to What You Need
- 10 Years of ‘Her’: What the Film Got Right about AI, What It Did Not
- Susan Wojcicki: The Screaming Legacy of The Quiet Architect of the Digital Age
- Apple Intelligence – Steve Jobs’ Company Finally Bites the AI Apple
- PSEUDO AI: Hilarious Ways Humans Pretend to be AI to Fool You
- Data Scraping AI Companies & Writers Fight to Define Future of AI
