Creator of "AI Stefanie Sun": Videos have been viewed 10 million times, and tutorials sell 3 times a month

The popularity of "AI Stefanie Sun" reflects the "emergence" of AIGC's technology in the field of voice - from ChatGPT to midjourney (the largest AI painting platform), AI is immersed in conversations, paintings, and now music. This brings many new thoughts to the industry and consumer market - how was AI Stefanie Sun created, what are the creators' demands? Will this change the production and consumption logic in the field of music? Let's take a look.

During the May Day holiday, Stefanie Sun, who had not been seen for a long time, appeared at the Changsha Haichao Universe Music Festival. At the same time, "AI Stefanie Sun" became popular on major short video platforms. Real people and AI echoed each other in different dimensions, and Stefanie Sun dramatically became the protagonist in the AI ​​field, which had little to do with her.

Songs such as "Mohe Ballroom" (original singer: Liu Shuang) and "Hair Like Snow" (original singer: Jay Chou) were sung by "AI Stefanie Sun", and netizens commented that they "reacquainted themselves with these songs."

The comments of the AI ​​Stefanie Sun version of "Hair Like Snow"

At present, the AI ​​Stefanie Sun's covers on Bilibili have reached millions of views and thousands of comments. The "AI Stefanie Sun" topic on Douyin has also exceeded 14 million views. Sorted by the number of views, the top video has nearly 7,000 comments. One listener commented that when "AI Stefanie Sun" opened her mouth, the song became "more advanced".

Audience comments on "AI Stefanie Sun"

"AI Stefanie Sun" has become a content label and a hot spot for self-media creation. Whether it is a music vertical account or an entertainment marketing account, whether it is reposting or making your own content, they all began to participate in "AI Stefanie Sun" and share the traffic dividend.

The endless emergence of "AI Stefanie Sun"

Through interviews with many behind-the-scenes creators, "E-Commerce Online" learned that some people joined in because they loved Stefanie Sun and "generated electricity with love", while others used it to make a profit, either by selling courses or "making accounts" - but the results were not great, with only three related technical tutorials sold per month.

At the Haichao Universe Music Festival, a fan asked Stefanie Sun about her opinion on the "AI Stefanie Sun" issue. She only smiled and replied that she had heard of it, but did not make a clear comment. Her agency clarified that it has not yet commissioned a lawyer to handle related matters.

As a phenomenal event, the popularity of "AI Stefanie Sun" reflects the "emergence" of AIGC technology in the field of voice - from ChatGPT to midjourney (the largest AI painting platform), AI is immersed in conversation, painting, and now music. This brings many new thoughts to the industry and consumer market - how was AI Stefanie Sun created, what are the creators' demands? Will this change the production and consumption logic in the field of music?

1. Stefanie Sun’s fans, generating power with love

How was "AI Stefanie Sun" created? From a technical perspective, this is an AIGC content product (AI generated content) based on a "generative speech model".

The creators "fed" the model with Stefanie Sun's voice material for timbre training, and then performed model deduction and timbre replacement on the pop songs they wanted to perform. To put it simply, it is to convert one person's voice into another person's, but retain the melody, rhythm and emotion of the original voice, which is a "virtual cover".

Strictly speaking, this should be considered a "deep imitation" rather than the so-called AI version of Stefanie Sun. There is also a consensus in the industry that no matter how close the AI ​​version of Stefanie Sun's voice is to the original, it is ultimately just a folk imitation show, with an indelible objective distance from the real person. A more complete way of saying "AI Stefanie Sun" should also be "the AI ​​imitation version of Stefanie Sun."

"I am a fan of Stefanie Sun. After hearing about the works of 'AI Stefanie Sun' on the Internet, I also wanted to hear my idol sing some songs she has never sung before, so I dove in." Han Long is a science and engineering student who is keen on "secondary creation" in the field of music. "In the past two years, I have tried some 'ghost animals' and accumulated some knowledge and skills. 'AI Stefanie Sun' and 'ghost animals' are actually similar to me, both of which are secondary processing of music."

The threshold for creating "AI Stefanie Sun" is not high. The most mainstream production tools are "UVR_v5.5.0" and "So-VITS-SVC 4.0". There are a large number of public "integration packages" on the Internet. Users do not need to understand technology or write algorithms manually. They can directly use the products built by predecessors.

So-VITS-SVC 4.0, the integration package has been designed to be quite "foolproof"

The process of organizing audio training materials is relatively time-consuming and labor-intensive, and has some technical barriers. "You need to prepare some 'dry voices', which are pure human voices without music, but Stefanie Sun's songs all have accompaniment and intervals, so you need to know some basic audio editing, such as extracting human voices and cutting audio into smaller, 'cleaner' clips."

This is to maximize the quality of voice training materials and get the best training results. "After training, import the song you want to replace the voice with and perform model deduction. If the voice is not similar, you need to increase the scale of the 'feeding' audio, continue training, and adjust the parameters until you are satisfied."

Those who can complete the above operations are actually considered "technical people". A more "novice" approach is to download an integrated package of sound models from the Internet. Users do not need to find additional materials or training, they can directly deduce and generate songs.

"As long as you have a little learning ability and a spirit of exploration, and a computer with decent configuration, and follow the tutorial step by step, you can create your own 'AI Stefanie Sun'." But Han Long also added that if you want better and more personalized results, you have to train the model yourself.

“You need a computer equipped with an N card (NVIDIA series graphics card). It takes about 30 to 60 hours to run a model (algorithm learning input audio material). It takes about half an hour to two hours to make a song.” Yi Ren is also a fan of Stefanie Sun. He added that if an integrated package with a trained model is used, it can help creators save at least 30 hours of training time.

"If your computer configuration is not good enough, you can also rent some cloud computing power." Nowadays, domestic cloud computing platforms of all sizes have launched cloud distributed GPUs for AIGC needs, with the lowest price being only a few cents per hour. "The more powerful the graphics card, the higher the training efficiency, but of course the more expensive it is."

Quotations from some graphics card rental platforms

According to a rough calculation, the cost of renting computing power to train an "AI Stefanie Sun" voice model starts at around 100 to 200 yuan, which is still considered cheap. So, an interesting question is: with such a low threshold and so many singers, why is "AI Stefanie Sun" the most popular?

Rcell, one of the original creators of "AI Stefanie Sun", told reporters that he and his team conducted hundreds of experiments in half a year and collected about 100 songs from Stefanie Sun's four albums, "Stefanie Sun's Self-titled Album", "Kepler", "Against the Light", and "It's Time", as training data. "We have tried to convert the timbre of other singers, such as Jay Chou, JJ Lin, Faye Wong, etc., but the results are not as good as Stefanie Sun. Stefanie Sun's voice is clear, stable, and distinctive, and her songs are diverse in style, difficult, and emotional, so they are particularly suitable for AI training."

“The more Stefanie Sun is trained, the higher the quality of the AI’s voice will be, and the more likely it will be sought after by the market. The first-mover advantage will snowball.” Yi Ren emphasized that in the current trend of AI versions of singers, the quality of “AI Stefanie Sun” is top-notch.

2. A highly profitable business, but few buyers

The most played video on Han Long’s WeMedia account is the “AI Stefanie Sun” version of “Qingming Rain” (original singer: Xu Song)
, reaching 267,000 views. The second most viewed video of the action figure only had 18,000 views, which is a big gap. "AI Stefanie Sun" can bring a lot of exposure to his account, but he said that he currently has no commercial or monetization needs. "Most of the people who participated in the creation of 'AI Stefanie Sun' do not have a clear business plan. It is purely for fun, but there are some possible monetization directions, and some people are indeed doing so. Those people are speculators, different from us."

For example, selling production tutorials and packaged training audios. The reporter observed that some bloggers would provide text guidance in the comment area below the "AI Stefanie Sun" content, or add external links to encourage viewers to buy tutorials and audio materials, with prices ranging from a dozen yuan to hundreds of yuan.

"The things they sell are available for free online, and they are not hard to find." Han Long explained that these businesses are not making money from "AI Stefanie Sun", but from the information gap that exists every time AIGC becomes popular. "Some users are not good at using search tools, or they may just be lazy."

Claiming permanent one-on-one tutoring for only 93 yuan

The reporter added a merchant who sells training audio and tutorials. His circle of friends also sells AIGC painting model training materials and ChatGPT account registration tutorials. "It's unsustainable and the sales scale is not large." The merchant admitted that doing this business is like "a blind cat waiting for a dead mouse" for him. In fact, there are not many orders, but the advantage is that there is no cost, all profit. "I sold two orders in a month and made more than 100 yuan, but I am quite satisfied. After all, I 'picked up' it."

E-commerce Online has previously described people who make money from ChatGPT, such as those who register 200 accounts and sell a system for 50,000 yuan. Looking back at the situation of those "middlemen", whether due to external supervision or their own reasons, the current ending is without exception, "leaving the market after making a profit".

Some people also rely on "AI Stefanie Sun" content to increase their followers, and then package the accounts for sale. In the field of self-media, this is a rather classic marketing idea - create content that is popular, do not consider long-term operations, and pay more attention to short-term traffic explosion and the speed of increasing followers.

More than 6,000 fans after more than a dozen videos

The reporter observed that there is an account named "陈墨瞳1995" on Bilibili, which has more than 6,000 fans thanks to more than ten "AI Stefanie Sun" videos. The accounts with more than 100,000 fans on Douyin that create "AI Stefanie Sun" content mostly have divergent works in previous periods - from AI singers to classic old songs, and even funny videos and Internet skills tutorials, which makes it difficult to find the main focus.

"In the algorithm of short video platforms, individual content has independent recommendation weights. Even if the quality of an account is average, if the content is on the hot topics supported by the platform, it can still receive considerable recommendations. Moreover, a lot of content may be pseudo-original, just modified, and the cost is low," an advertising operator who used to work for a leading domestic short video platform told reporters. If the platform has tens of thousands or hundreds of thousands of fans, it can be transferred to middlemen for sale. "Some small business owners follow the trend and create a company Douyin account, so they will think about buying an account with a basic number of fans. There is still a market for it, and it doesn't cost much anyway, just a few thousand yuan will do."

But he also admitted that the basic weight of this type of account is limited, and the traffic depends entirely on the quality of the content published subsequently, so it is actually an "empty shell."

Of course, companies with stronger technical and marketing capabilities had already packaged such models and algorithms into consumer-grade apps long before "AI Stefanie Sun" became popular, allowing users to input the sounds they want to train, select the songs they want to replace, and then generate a cover version with stable quality with one click. Domestically, clients such as "Virtual Singer", "ACE Virtual Diva" and "Geji Geji" all have similar functions.

But before making money, businessmen who devote themselves to the creation of "AI Stefanie Sun" may need to consider the issue of "compliance" more.

3. Supervision and support are both "encouragements" for the industry

On May 9, Douyin released platform regulations and industry initiatives on AI-generated content. The main measure is that publishers must be responsible for the corresponding consequences of AI-generated content. At the same time, publishers should clearly mark AI-generated content to help other users distinguish between virtual and real, especially easily confused scenes. At the same time, similar supervision has also appeared on the Xiaohongshu platform. After being identified as AIGC by the system, the corresponding logo will appear above the content.

Current major regulatory measures for content platforms

The platform took the initiative partly because AIGC is still in the "gray area" of copyright, and the recently popular "AI Stefanie Sun" is particularly controversial because it involves strong copyright elements such as stars and songs. Previously, an AIGC song "Heart on My Sleeve" that synthesized the voices of singers Drake and Weekend was launched on the Youtube platform. But then, the singer's record company Universal Music asked the platform to remove the song and asked, "Are you on the side of artists, fans and human creative expression, or on the side of Deepfakes, fraud and refusal to pay compensation to artists?"

According to foreign media Gizmodo, a netizen used AI to generate counterfeit songs of internet celebrity singer Frank Ocean on Discord software and defrauded $13,000 by selling these forged songs; but on the other hand, Canadian singer Grimes publicly declared that he could accept others using his voice to synthesize songs, provided that he was paid half of the corresponding profits.

Concerns about copyright actually emerged long before AIGC Music was launched on a large scale. The "So-VITS-SVC 4.0" product interface has the following warning:

Yao Huanqing, associate professor at the School of Intellectual Property of Renmin University of China, said: "First, it involves the infringement of the singer's right to name. Without the consent of others, saying that this is AI Stefanie Sun or AI Faye Wong is equivalent to borrowing the singer's commercial reputation to a large extent. At the same time, the use of other people's legally recorded music works to produce sound recordings does not require the permission of the copyright owner, but remuneration should be paid in accordance with regulations. If A's timbre is used to sing B's song, the copyright of B's ​​song is involved."

The fact is that most of the sound materials used for model training do not come from legitimate recordings, but from the Internet, so the copyright is difficult to confirm. "If the material itself is illegal, then there is an infringement issue," Yao Huanqing added.

Unlike other platforms that have increased supervision, Bilibili launched the "Virtual Voice Creation Plan". The platform encourages creators to "create original works, cover tunes, write lyrics, and adapt songs" according to requirements, and provides traffic support and rewards. So far, this activity has been viewed nearly 100 million times.

Supervision and support are not contradictory. They both aim to promote the activeness of the AIGC creation market in a more "compliant" way. For reference, in the field of AI painting, there is a trend of artificial painters adding "digital tags" to their works to prevent the works from being learned and trained by AI without authorization. This move may also be used as a reference by original singers in the future.

4. The future of music: refined or “rough”

In 2017, singer Taryn Southern collaborated with artificial intelligence software to release the album "I AM AI". People involved in the project predicted that "the future of music will be created between the collaboration between humans and artificial intelligence."

I AM AI

In 2021, South Korea launched a program called "AI vs Human". In it, AI imitated Kim Bum-soo's song "Miss You" in the voice of the late singer Kim Kwang-seok. "Miss You" was created in 2002, and Kim Kwang-seok died as early as 1996 - AI "resurrected" a departed singer and sang a song that he could not have sung.

In China, in 2022, Kugou also teamed up with Yang Chaoyue to launch the "AI Singer Yang Chaoyue", involving more than a dozen AIGC songs, using technology to make up for Yang Chaoyue's singing skills.

AI has already fully participated in the field of music creation in terms of composition, lyrics, and even singing skills. But the issue behind this that is more worth thinking about is the potential trend of "AIGC's reconstruction of content production and consumption."

As creative tools become more and more "idiot-proof" and thresholds and technologies are decentralized, content production has moved from PGC to UGC and finally to AIGC, and content creation has entered the "era of the whole people". The status of singers, arrangers, lyricists, and record companies in the upstream and downstream of the industry may face a "rewrite". In the past, singing skills were considered an art. But with the popularity of "AI Stefanie Sun", singing skills seem to have a tendency to become a replicable "methodology".

At this point, singers need to rediscover their own "barriers". In this regard, a thought-provoking story is that in the music program "The Sound of Life", music critic Ding Taisheng praised the performance of singer Yang Naiwen. He particularly pointed out that "Yang Naiwen missed a high note at the end of the song. Although it is a flaw, it is also the most touching part. People nowadays have an almost pathological obsession with skills and techniques, but singers should have a human flavor, otherwise all the singing work can be handed over to AI."

It can be seen that in the past, singing was more like an art that pursued "refinement", and the industry pursued higher high notes, more stable and accurate voices, and more unique timbres. However, under the "competition" of AI in the music industry, refinement seems to no longer be a human's strong point. Some "mistakes" in performance are more touching because they are closer to "humans".

In addition, when singers become "nutrients for AI", if we look at this matter positively, the creation of personalized AIGC will undoubtedly drive the diversification of the content market and indirectly increase the popularity of the singers behind it. But correspondingly, singers also run the risk of "losing focus" of influence. The industry is no longer chasing the "person" of the singer, but only his voice. Therefore, for the music industry, in the future, it is necessary to find the real "difference" between humans and AI.

Author: Shen Songnan; Editor: Si Wen

WeChat official account: E-Commerce Online (ID: dianshangmj), see sharpness and insights, and focus on innovative media on the Internet and new business.