"AI Stefanie Sun" becomes a top streamer on Bilibili, with millions of views

AI voice-changing technology has recently set off a wave of popularity on video websites. Here, Stefanie Sun can "sing" Hair Like Snow and Jay Chou can "sing" Yu Jian. Many people, while feeling the novelty, can't help but worry about the career future of professional singers. The author of this article dispels such doubts and popularizes the basic principles of AI voice-changing, and makes a positive outlook.

In the next stage of AI development, will singers face unemployment in advance?

No one expected that the "unpopular singer" Stefanie Sun would be so active recently, with her cover works on Bilibili receiving numerous reviews. But when fans came to her website, they found that the screen was full of "AI Stefanie Sun".

It turns out that these so-called Stefanie Sun's cover songs were created by B Station UP hosts using AI technology , and were not sung by Stefanie Sun herself.

Other similar works include "Qinghai-Tibet Plateau" sung by AI Wang Xinling, "I'm Sorry for Being So Cute" sung by AI Xu Song, and "Itchy" sung by AI Rihanna. The styles of the singers and songs are very different, but the performances are almost harmonious.

Such a lively scene made netizens marvel at the power of technology while worrying about their idols - will singers also be replaced by AI?

01 How popular are “AI Stefanie Sun” and others?

How popular is the AI ​​cover song?

When searching for "AI Stefanie Sun" at Bilibili, the most popular one is her cover of Jay Chou's "Hair Like Snow", with 174.6W views, over 2,000 comments, more than 40,000 likes and reposts, and 22,000 coins from UP hosts.

On the homepage of this UP host, we can see that before the release of AI Stefanie Sun's related works, the number of views of other content was only a few hundred or a few thousand. Since the release of AI Stefanie Sun's cover songs in April, the number of views has basically exceeded 10,000, and there are even some contents with around 100,000 views, which shows the popularity of this type of content.

In the search results, the second and third most played content came from the same UP host, who respectively covered Jay Chou's works "Peninsula Iron Box" and "Love in BC", both of which have been played over a million times. Take "Peninsula Iron Box" sung by AI Stefanie Sun and Jay Chou, for example, it has 47,000 likes, 29,000 coins, and more than 20,000 favorites and reposts, which can be considered a popular work.

It is worth noting that the UP host has only 40,000 fans, and the turning point of the content data also began when the AI ​​Stefanie Sun’s abnormal content was released. Its update speed is fast, and the number of views of many contents can reach hundreds of thousands, including not only AI Stefanie Sun’s solo, but also duets with singers such as Jay Chou and JJ Lin.

In addition, the AI ​​Stefanie Sun on Bilibili can not only sing rap, but can also be combined with AI face-changing videos.

UP host "Raccoon Leon" mentioned in the video description that "the original video and basic audio excerpts of the redrawn SD video are from the cover of Li Yugang's "Wanjiang" by the B station master: Xiu Xiuman". The face was generated by AI imitating the real UP host, and the voice was produced by AI according to the tone of Stefanie Sun, but except for a slight pause in the dynamics, the whole video is very realistic and has received 178,000 views.

In addition to cover songs, data on content related to AI Stefanie Sun are all at a good level. For example, a video listing AI Stefanie Sun's hit songs has 705,000 views, and a video interpreting this phenomenon has more than 200,000 views. Most tutorial videos easily exceed 10,000 views, and quite a few have reached over 100,000 views.

It can be said that AI Stefanie Sun has indeed created a craze on Bilibili. There are only collaborations that fans cannot think of, and no songs that UP hosts dare not try. After all, who would have thought that they would be able to hear "Stefanie Sun" singing "Hero Song" in their lifetime?

02 Can “AI Stefanie Sun” take away Stefanie Sun’s job?

In the process of "AI Stefanie Sun" becoming popular, we can see that its promoters are not just a single group of people.

Among the fans of AI cover songs, one is the singer's voice. " Unpopular singers" have been out of business for a long time, so why not make their own music? In addition, Stefanie Sun is famous for her "diamond voice" and can match more songs, so AI Stefanie Sun stands out from many AI covers.

The second group is AI technology enthusiasts, whose content is updated as technology iterates, and whose works are just touchstones of technology.

In addition, there are followers who want to reap the traffic dividend while the trend is hot, as well as Bilibili itself.

Bilibili provides support for AI-related content. Its "Virtual Voice Creation Plan" campaign calls on users to use various sound sources to create.

According to Weiguojiang’s observation, many AI Stefanie Sun’s videos carry the tag #AI虚拟之声实验室#. The videos under this topic use technologies such as RVC, DDSP-SVC, DiffSinger, Diff-svc, and SoVITS-svc to train or apply AI voice models, create songs or dub, and submit them to the [Music-VOCALOID·UTAU] section.

Many AI Stefanie Sun uses an open source vocal cloning model, which outputs cover works through three steps: inputting vocal samples, training models, and model inference.

The vocal samples need to be the original voices of the characters with lossless sound quality. In other words, the model cannot be fed with just ordinary songs, but the singer’s voice after separation and extraction, and as many samples as possible should be uploaded to form a data set. Before training the model, the songs also need to be split into files of about 10 seconds and then imported to ensure the speed and quality of model training.

When it comes to the model inference step, the vocal part of the target cover song needs to be extracted in advance, and after importing the trained model, the vocal part of the AI ​​cover can be output. Finally, you only need to use editing software to add accompaniment and align, and an AI cover work is produced.

Screenshot from: B station UP host "AI next generation"

It is not difficult to see from the above steps that the threshold for making an AI cover video is not high . The tools used in the process can be found in open source or free software. As long as the hardware can run, even a technical novice can follow the rich tutorials on the Internet step by step.

This has also sparked a discussion among netizens about whether singers can keep their jobs.

In fact, for now, the singer's job is still secure.

The first is the limitation of technical level.

Although the tutorial looks simple and anyone can get started, it is not so easy to actually output the work. Factors such as the adjustment of model parameters and the different human voice samples fed will affect model training and reasoning. The final generated results may not be completely as expected, and there is great uncertainty.

In addition, most of the current AI covers only replace the timbre, and the singer's own skills and injected emotions are not reflected, so many fans have criticized it for problems such as unsmooth performance and inconsistency in details. Of course, such statements still cannot stop everyone from affirming AI covers.

The second is the influence of subjective emotional factors.

Listening to music is an emotional thing. Whether it is the love for the singer himself or the story of the time and place, these are emotional values ​​that AI cover songs are difficult to provide.

So, rather than worrying about singers losing their jobs, it is better to say that AI will squeeze out the space for cover singers to perform.

As AI technology becomes more mature and cover songs are closer to the singers themselves, cover singers who originally relied on this to attract attention but did not have the ability to create songs may have a stronger sense of crisis. After all, in the imagination of netizens, the future song software will be able to choose songs and singers.

On the other hand, AI technology is also a new practical tool for singers.

In March of this year, singer Chen Shanni collaborated with the Taiwan AI Labs team to release a love song titled "Teach Me How to Be Your Lover" sung by an AI trained by herself. The rhythm, pauses, single cover, etc. were all generated by AI, and the degree of realism sparked discussion.

Such a work is a manifestation of the singer's embrace of new technologies, and is also one of the aspects of AI as a double-edged sword.

Some singers are willing to spend more time to improve AI's singing skills, while others choose to use AI to create meaningless "brainwashing songs" because as long as the number of samples is sufficient, AI can grasp simple but catchy melodies in minutes. For the audience, such AI songs will make the music market even more chaotic.

Before the unknown future arrives, the greater risk AI faces in the field of music is copyright. Just as one of the reasons for resisting AI painting is the copyright of the samples used to train the model, AI cover songs have also been questioned for infringement.

In an analysis article published by Beijing Zhide Law Firm, it was mentioned that AI cover songs may involve infringements of copyright, voice rights, personal information, etc. However, since there are no specific laws and regulations in China, most AI Stefanie Suns have attached a disclaimer and continue to actively update on the platform.

In contrast, the foreign music market has already responded.

AI Hub, an AI music production community with more than 20,000 members on Discord, once produced an AI music album, but it was marked as infringing and removed by Warner Music three hours after it was released. In addition, among the singers, some called for a boycott, while others said they would share the profits with cover singers.

In summary, the AI ​​Sun Yanzi's carnival may not really shake the cheese of singers, but the impact on the music track does exist. It is still unknown whether the end of AI cover will be a lawyer's letter or a new profit channel for singers like short video slices.

What is certain is that the impact of AI technology is expanding and deepening. In every industry, some people are replaced, while others use it as the strongest auxiliary. As Chen Shanni said on Weibo, if the era of AI is bound to come, as creators, what we should care about is not "will we be replaced", but "what else can we do".

Author: Chen Chumu

Source: WeChat public account "Weiguojiang (ID: wjam123456)"