AI tools have not yet been used for short dramas

From Kuaishou's "KeLing" to Byte's "Jimeng", AI short dramas seem to have stood at the forefront. This article deeply analyzes the current situation and future of AI short dramas, explores how AI can save costs and improve efficiency while maintaining the originality and high quality of content, and takes readers to glimpse the infinite possibilities of AI in the field of short dramas.

During the early AI video testing, AI short play director Zhao Kang (pseudonym) accidentally discovered a bug in Keling, saying that the Wensheng pictures generated had a Kuaishou style. At that time, there was only one sentence in his mind: "My friend, go to the link."

Xiao Lei (pseudonym), an AI video creator, has tried all the tools on the market, but still can’t put together a usable set of tools. “AIs come and go, but MJ (Midjourney) remains.”

"The motion brushes of PixVerse, the first and last frames of Bytes are Dreams, and the logical understanding of Ke Ling", AI creators have gradually figured out some tricks.

As for where AI videos can be used, big companies and creators have all targeted the short drama track.

Kuaishou took advantage of the popularity of its self-developed video generation model "Ke Ling" and released the trailer of "Shan Hai Qi Jing: Breaking the Waves" early. The excellent production attracted all the attention. However, it was Douyin that actually grabbed the "first AI short drama" prize. On July 8, "Sanxingdui: Future Revelation" (hereinafter referred to as "Sanxingdui") produced by Bona Film Group and Douyin AIGMS Production Center was launched.

On July 12, Kuaishou Xingmang short drama "Shanhai Qijing: Piercing Waves" (hereinafter referred to as "Piercing Waves") was launched. Photon Planet learned that Piercing Waves had completed the last round of revisions in mid-June, but director Chen Kun insisted on using the latest model optimization of "Keling". "I hope that my work will represent the highest level of the industry."

Even though AI short dramas have created a lot of hype, they are still just a gimmick. AI still has limited efficiency improvements for the entire short drama production process, and the cost and production cycle have even increased instead of decreased. Although AI tools are infiltrated in every link, they are like shells on the beach, requiring creators to spend a lot of effort to pick them up.

"If you perfect one or two functions, there will be a market for them and they can be commercialized," said Chen Kun.

1. The “account book” of AI short plays

New technologies are running forward blindly. So far, the industry has not yet clearly defined AI short dramas or AI short films. At present, there are two sources of AI short dramas: one is the change in the short drama production model, and the other is the iteration of AI video tools.

The production of short plays mainly involves script writers, contractors, and MCN organizations. It takes 5-8 weeks from text creation to release. A Liang has created many short plays in 2021, including some hits with tens of millions of viewers. He told Photon Planet that it takes about 4 weeks to create a 100-episode short play script of about 50,000 words. "Some assembly line short play companies that exploit employees have multiple people writing scripts, and can produce one in as fast as two weeks."

When the script is ready, investors or MCN organizations will inject funds and look for producers to produce the show. Wang Xian (pseudonym) is one of the producers. He has participated in many short dramas. This year, he participated in the production in multiple capacities, including "independent producer" and actor. "In the early stage, we communicated with the script and coordinated with the actors. It takes an average of 3-7 days to enter the shooting stage. The shooting time is about 7 days, and most of the time is wasted on post-production."

In addition to the time cost, the production cost of a short play varies depending on the subject matter and the level of sophistication. Aliang has collaborated with top MCNs such as Jiuzhou on many short play scripts from 2021 to 2023. The average price of buying out a script is 10,000 to 20,000 yuan. Some companies will sign a clause with the script author to receive a commission for the hit. "The most popular play, plus the commission, received a total of more than 300,000 yuan."

Manpower accounts for the bulk of the cost of short drama production. Taking Zhengzhou as an example, the cost of the main creative staff accounts for a very high proportion. The average daily salary of the male and female protagonists is around 3,000 yuan, and the extras are 150 yuan per day (including the commission of the middleman). If post-production is included, the overall cost is between 400,000 and 1 million yuan.

"Since the beginning of this year, the division of labor on site has tended to be similar to that of TV drama shooting. Previously, there were directors, actors, cameramen (1-3 cameras), costume and makeup artists. This year, there are dedicated prop masters, and some short dramas even have professional stage controllers." After the shooting is completed, it will be outsourced to a third party for post-production, with a period of 1-2 weeks.

In theory, AI can reduce costs and increase efficiency when it comes to short drama production, but we got completely different answers from two senior AI video creators. At least for now, whether it is based on Kuaishou KeLing, Byte Dream, iPoetry PixVerse, or mixed with video tools such as Runway and Luma, the overall cycle and cost are higher than traditional short dramas.

AI artist Chen Liufang has almost completely experienced the evolution of AI video tools. The first technology she came into contact with was called VQGAN (training Transformer to achieve high-resolution image synthesis), followed by CLIP guide Diffusion (generated through Clip-guided diffusion model), which is the predecessor of Stable Diffusion (hereinafter referred to as "SD"), which was born a few months later.

AI short dramas or short films are still in the stage of setting benchmarks and exploring production paths, and it is difficult for us to quantify the complete production cost of a film.

"Chopping Waves" is a short drama customized by Kuaishou. During the production process, the Keling team coordinated a lot of technical resources to co-create with director Chen Kun's team. An insider said, "We provide technical support to Chen Kun, after all, he is Kuaishou's 'son'."

On Valentine's Day in 2023, Chen Liufang used Stable Diffusion as the basis, using AI to create the entire process from creativity, script, storyboard, animation production, and music, and completed a one-minute short film "Earth Love Finale". The second short film was born in August, adapted from Arthur Clarke's science fiction novel "Song of the Distant Earth".

Chen Liufang has currently formed a professional AI short film creation team, recruiting creators who have both traditional animation creation skills and the ability to master AI tools. The team has also received external support from technical experts from many large companies.

This year, large video generation models have emerged in large numbers both at home and abroad, so the current AI video presentation shows that "the upper limit of the large model manufacturer's model is the upper limit of the creator."

At present, there are many differences between AI short film production and traditional short dramas on the market, and the difference is huge in terms of money and time alone.

According to a survey conducted by Photon Planet, the production cost of AI short films fluctuates greatly depending on the quality requirements. The production cost of a single episode can be as low as 10,000 yuan or as high as 1 million yuan. Usually, it is divided into three levels: movie-level head and quasi-head quality at 100,000 yuan per minute, traditional animation content at an upper-middle level at 30,000-50,000 yuan per minute, and educational animation and ordinary animation can be as low as 1,000 yuan. Compared with the average 10,000 yuan per episode of short dramas, the production cost of AI short films is obviously much higher.

It is worth mentioning that a large amount of the investment cost is a one-time investment, so it is difficult to share it through reuse. In terms of time cycle, a short film of less than 10 minutes takes about 1 month. Chen Kun's earlier AI promotional film "Feng Ming Shan Hai" for the film channel M Festival also took half a month. The long cycle and high cost make the real "financial sponsor" of AI short dramas actually come from the G end.

"G-end customers' enthusiasm for new technologies is much higher than we imagined," said Chen Liufang.

Technology is in a process of continuous iteration, and it is bound to cause a series of bugs and other problems when it is implemented. Zhao Kang has gone through the entire process of research and development, implementation, and commercialization. When we asked him what to do when he encountered a technical bottleneck, his answer was: "Convince the client." After a pause, he added: "When you can't solve a problem, solve the person who raised the problem."

2. The Transformation of AI in Drama Production

Many creators, including Chen Liufang, believe that even in the AI ​​era, the market's standards for good content remain unchanged. Therefore, the underlying driving force behind an AI short play or short film still comes from a good story and a good idea.

In the stage of inspiration and script writing, AI tools with "novel writing" as the entry scenario have appeared on the market.

According to the self-media "Buzhi" on AI writing, creators previously spent a lot of time collecting materials. In order to write an ancient Chinese article, an author needed to collect more than 100,000 sentences of materials in the early stage. But with AI, Buzhi felt a significant speed-up. A short article that used to take more than a week to complete can now be completed in two days, even without inspiration.

Wan Lei, CPO of AI startup Waveform Intelligence, believes that the novel writing scene is a "hub area", which moves forward from text creation to transformation into scripts and short plays, thereby "opening up the entire chain of literary creation."

"Wa Wa Writing", a product under Waveform Intelligence, is a product for content creators. Official data shows that as of July, the Wa Wa Writing platform has accumulated 240,000 registered users, with an average daily usage time of 4.7 hours, and AI has cumulatively output 10 billion words. Recently, the platform upgraded to version 2.0 and added a new "novel to video" function, which first disassembles the main characters in the article, generates the image of the character, and then produces a video with storyboards, trying to push creators to extend to the direction of AI short dramas.

For short drama/short film creators with professional backgrounds, they prefer the production process of "text to image" to "image to video". Although this path has been run, the role of AI is still limited.

At present, in the field of AI short dramas/short films, the themes can be roughly divided into the following: fantasy/science fiction, realism/semi-fantasy and semi-realism. Generally, AI video creators tend to start with fantasy/science fiction themes, which not only alleviates the problem of heavy assets and heavy investment in traditional science fiction themes, but also fits the imaginative technical tone of AI. Compared with realistic themes that are closer to expressing human emotions, the audience is more tolerant of science fiction themes that break the boundaries of imagination.

Chen Kun insisted on using "pure AI" to create the short film from the beginning. However, he admitted: "AI can only partially replace the original production process."

The traditional film and television production process is divided into pre-production (creative discussion, script writing, etc.), mid-production (preparation, shooting preparation, shooting, on-site management, etc.) and post-production (editing, visual effects, color correction, music, etc.).

In the early stage, the director has to meet with the screenwriter to discuss the script, and then the director will make a storyboard script. AI can only play the role of a "plug-in" at present, such as using ChatGPT to expand ideas and check whether there are text errors. "Core creative work still relies heavily on manpower."

The most profound AI replacement is in the mid-term stage. In the past, the director was responsible for coordinating and putting together a team of lighting, sound, camera, actors, makeup artists, and costume designers, and then filming the actual content. Now, as a director, you only need to use Midjourney, Stable Diffusion, and domestic and foreign AI video generation tools to directly generate pictures from text, and generate video clips from pictures.

"Before, as a director, I had to tell the producer which location to choose, whether the lighting should be direct or side backlighting, and how the camera lens should move. Now, all of this has become telling the AI," said Chen Kun.

Although AI video companies in the current market claim that they can generate several minutes of footage, in actual operation, they can only generate 3-5 seconds of usable material. In other words, a three-minute film needs to generate at least 36-60 clips, provided that it is usable. A common situation is to jump back and forth between software such as Keling, Jimeng, PixVerse, Runway, and Luma, and keep drawing cards until satisfactory material is generated.

The highlight of AI short dramas/short films lies in the post-production. The production team needs to edit dozens of clips together, including music, special effects, subtitles, sound effects, and color grading. If the team finds that there is a lack of material during the editing process, they have to go back to the previous step. Take the Arctic Ocean advertising short film produced by Chen Kun as an example. It was made purely by AI, and the film was more than one minute long, and it took two weeks including post-production.

Zhao Kang told Photon Planet that the post-production of AI short films accounts for as much as 80%-90%. Under the extreme operation of traditional short dramas, each episode is five minutes long, and a total of 35 episodes of short dramas can be completed in three days, filmed in four days, and released online after editing in a week. In comparison, the AI ​​short drama industry is "both motor-consuming and power-consuming."

If AI is used to present films that are close to real human life, the difficulty and cost of production will only be higher. Chen Liufang made an attempt to explore the boundaries of technology in "To My Dear Myself".

Its responsibility is reflected in the process of filming and training models. Chen Liufang introduced that after getting the original story, it was first adapted into a film and television version. While filming in the green studio, her team was also training the required role models and style models. "A role changing a set of clothes is a model." In order to achieve better results, she trained nearly 20 models. Part of the content shot in real life is entered into the post-production software for synthesis, and part is used for scene construction. Finally, the preliminary video screen is obtained through the AI ​​transfer function.

After the transfer, some repairs are still needed. Chen Liufang said, "About 10% of the shots require manual adjustment, such as small objects and delicate character performances."

AI has raised the difficulty of short film production to another level. Bringing together the top domestic directors and AI creators, the production cycle of "To My Dear Myself" lasted two and a half months, and it eventually won the Best Film Award in the AIGC Short Film Unit of the 14th Beijing International Film Festival.

3. From plug-ins to tools, AI video will eventually move towards integration

Chen Kun has a habit of remaking an old work whenever a new AI video tool appears. When asked why he does things differently, Chen Kun just smiled and replied, "It's just that I'm skilled in using the tools."

AI video tool manufacturers are very keen to invite AI creators in the industry to join them. Chen Kun has worked with Aishi and Kuaishou respectively, and Ainimate Lab where Chen Liufang is located has just cooperated with Zhipu Qingying and Shengshu Vidu to release an AI animated short film.

These manufacturers hope to promote the effects that can be achieved through professional AI video creators, but upon closer inspection, they are full of "special offers for creators", such as more controllable platforms, additional technical guidance, and so on.

At present, domestic tools frequently used by AI video creators include Aishi PixVerse, Byte Dream and Kuaishou Keling. Zhao Kang told us, "If there are 100 functions for making videos, each company has only developed one or two."

It seems that all of them are promoting the length, clarity and accuracy of the generated content, but in fact, each one is different. In order to produce content accurately, creators are constantly testing the strengths of different tools.

"A task that is difficult to solve with tool A may become a one-click function in tool B," said Chen Liufang.

However, at the current stage, creators are still in the process of actively adapting to tools, and the implementation posture of native AI applications should be that tools adapt to people.

Based on feedback from multiple AI video creators, PixVerse is the first company in China to introduce the motion brush function. In response to the Motion brush of Runway Gen-2, PixVerse launched the Magic brush. For professional entrepreneurs, it means that they can have control over the content screen. Compared with the Motion brush, the Magic brush has a lower threshold for use and is easier to operate. There is no need to adjust complex parameters. You can control the movement of objects by just "dragging the arrow".

For creators like Chen Liufang, the "role lock" function of Pixverse brings her great convenience. Pixverse provides an entry point for creating new characters on the tool platform. Creators can specify characters to perform and generate shots. After locking the character, the consistency of the character's personality can be maintained.

ByteDance's Jimeng is like an average good student, maybe not amazing, but can get 70 points in every subject. Many creators said that before the emergence of Kuaishou Keling, they used Jimeng to make deliverable films for a long time. No matter how exaggerated the AI ​​video manufacturers' propaganda is, it can't beat the "usability" and "commercialization" in the workflow, which happens to be ByteDance's strengths.

Jimeng also has a unique feature: the first and last frame function. If you want to generate a 3-second video, you only need to upload photos of the initial and ending states, and AI will automatically fill in the changes in the middle.

The classic shot "Blue Luan Breathing Fire" in Chen Kun's "Phoenix Singing in the Mountains and Seas" uses the first and last frame function. The first frame inserts a picture of a bird circling in the air, and the last frame inserts a picture of a bird breathing fire, which naturally presents the process of the bird breathing fire.

The latecomer Keling started to build momentum and became the most popular. Official data showed that more than 500,000 people applied within one month of its launch.

Keling's understanding of the laws of physics impressed the first batch of test users. The core of this is the ability to "create something out of nothing" through prompts. For example, it can respond to prompts such as "raise your hand", "pick up the cup", "drink coffee" and so on based on the original picture to generate a new video that conforms to logic.

Zhao Kang revealed that he found that Keling is particularly suitable for making funny and imaginative films. "The success rate of drawing cards is quite high, about 30%-50%."

In the short term, the situation where creators wander around various AI video creation platforms may not change. First, each company has limited resources to invest, and second, the video creation process is too complicated to be fully covered. This actually gives latecomers an opportunity to catch up. In the field of AI, there is almost no "lying flat", and any company must be prepared to be "surpassed" at any time.

But if we look at it in the long term, the above anti-human operations will be overturned sooner or later. Referring to Adobe's idea, integrating single-point functions into a workflow software is the solution to achieve collaboration and reduce costs and increase efficiency.

In the future, a platform may be needed that can integrate the AI ​​video functions of Aishi, Keling, Jimeng, Shengshu, Zhipu Qingying, etc. Creators can call various functions on demand, and Agent assistants can also assist in production. Going further, the AI ​​video creation process is connected with the entire distribution ecosystem to achieve a closed loop of content creation of "production-distribution-streaming".

The above-mentioned platforms are like workshops. Creators provide raw materials, which are processed on the workshop production line. Finally, the finished products are put on the video stores for the market to choose from.

Written by | Hao Xin and Wu Xianzhi Edited by | Wang Pan This article is written by the author of Operation Party [Photon Planet], WeChat public account: [Photon Planet], original/authorized to be published on Operation Party, and any reproduction without permission is prohibited.

The title image is from Unsplash, based on the CC0 protocol.