The silent movie ‘Le voyage dans la lune’ (A Trip to The Moon), released in 1902, was considered the first science fiction movie in the world. Nothing like today, the movie’s ‘special effects’ were all human-made. 120 years later, we are standing at the crux of creative exploration, where AI can pretty much make a movie.
With the surge in AI video generation tools, the actual usage of these AI tools is yet to be measured in big production houses. Tools such as Stable Diffusion, Runway and the recent Pika may look promising at the moment, but there are still limitations that can restrict a creator from fully utilising them. However, production houses have already started experimenting with it, not just from a creative aspect but from an angle of operational efficiency.
AI Upgrades Pre-Production
While AI has been used at various stages of movie production, post-production (editing) is where the latest advancements in AI tools are implemented. From analysing clips for finding the best shot, splicing, de-ageing and other visual effects, including deepfakes, AI has found a crucial place to save time and effort. It has now found its way, in the pre-production stage , (planning) as well, where the biggest use case is storyboarding.
One of the crucial steps at the planning stage of a movie production is storyboarding. A storyboard visually outlines the sequence of shots that will unfold in a video. If done incorrectly, production houses not only face wastage of manpower but also face huge financial losses.
Realising the potential for simplifying storytelling and breaking barriers for those who wish to enter filmmaking with financial constraints and technical dearth, d.cult studio , a startup in Trivandrum, is out to ‘democratise the art of filmmaking’ through AI. The company is already working with studios in the Malayalam movie industry including Festival Studios and major production house Prithviraj Productions.
“What used to be sketches and infographic-based, is now changing into a cinematic-style storyboarding,” said Gokul A, founder of d.cult and co-founder of Accubits Technologies.
https://twitter.com/dcultstudio/status/1706578659195146533
Essence of Time and Cost
With accurate storyboarding, time and cost are saved during actual shoots. There’s a 50-80% overall reduction in time for the entire storyboarding process, and around 40-70% overall reduction in costs, after factoring in reduced labour and material expenses.
“In a 50 day shoot, where the per day shoot cost is 5 lakhs, even if five days are reduced, owing to AI storyboarding, 25 lakhs are saved from the production cost,” said Gokul. For a storyboard creator, such a tool is believed to augment their work. “Initially, where an experienced storyboard creator could make only two images in a day, such a tool will help the creator make upto 100 images in a day.“
The company uses a mix of models including proprietary and open-source ones, along with NLP algorithms. Idea to story/screenplay generation uses BUD LLMs – GenZ model, with natural intelligence algorithms. For auto music suggestion and background music integration, music generation models such as OpenAI’s Jukebox is used. Similarly, deep learning and computer vision models for AI video analysis and storyboard visualisation, respectively.
Treading Around Copyrighting
Storyboarding is only one part of the AI solution that d.cult studio offers. They also provide options of converting scripts to storyboard, collaborative whiteboard for idea discussion, video analysis for story ideas, pre-visualisation, photo generation, among other features.
With copyright issues causing a churn in companies producing AI-generated content, especially AI images, d.cult studio ensures strict measures on the same. “We are diligent in using only those materials that are either originally created by our team, fall within the public domain, or are available under creative commons licences for AI inputs,” said Gokul.
Animation Studios Are Not All Yay
Hayao Miyazaki- Studio Ghibli. Source: LA Times
While companies are experimenting with AI to ease movie production steps, there are some who will not jump on the wagon just yet. Signature style production houses such as Studio Ghibli, a Japanese animation studio that still uses traditional hand-drawn animation methods, will not embrace AI all too quickly. Co-founder and director Hayao Miyazaki has previously said that AI art is ‘utterly disgusting’ and that it will never succeed in giving artwork the genuine feel of a human being. He even said that he will never incorporate such technology in his work as he feels that it will be ‘an insult to life itself.’
On one end when you have animation studios shying away from embracing AI, there are others who are refining animation styles with the same. The LoRA training model simplifies the process of training Stable Diffusion on various concepts, including characters or specific styles. Once trained, these models can be exported for use in generating content by others. Thereby, enabling control levels for AI animation.
Furthermore, many are experimenting with a mix of models to create their own anime. Entrepreneur Varun Mayya said that production costs for an episode of anime costs from $140,000 to $180,000. He believes that making anime on their own with a mix of tools can probably drop the price by a 100 or 1000. He created one with a string of 20 different AI tools.
While people are experimenting with generative AI tools to create their own anime, movies, and more, the scaling of these techniques is yet to be seen. While human creativity will continue to play a significant role in movie production houses, thereby limiting complete adoption, the implementation of AI for operational and cost efficiency is where the actual use-case will lie.
