“AI Resurrection”, comfort or business?

In recent years, the topic of using AI technology to "resurrect" relatives and friends has been controversial. So what are the pros and cons of using AI to "resurrect"? Is it a technology used to comfort people, or is it a motivation for businesses to make money? You may want to read this article.

Some people rely on technology, some people rely on routines.

In the past, "resurrection" was something that was only seen in science fiction movies, but this year, there are more and more cases of "resurrection".

At the end of February, well-known musician Bao Xiaobai used AI to "resurrect" his daughter, who sang a birthday song for her on her mother's birthday; at the beginning of March, at SenseTime's annual meeting, Tang Xiaoou, the founder of SenseTime Technology who had just passed away, was "resurrected" in the form of a digital human and gave a "Tang-style" speech; recently, some netizens used AI technology to "resurrect" deceased stars such as Coco Lee and Qiao Renliang, which has been controversial.

"Meeting" with deceased relatives again, this previously secretive and niche business has begun to appear frequently in front of the public. However, due to different technologies and prices, the effects are also different.

For 10 yuan, you can make the photo of a deceased relative "blink", which is based on simple image processing technology; for a thousand yuan, you can have a video call with a relative who left without saying goodbye, which uses AI face-changing and voice-changing technology; for ten thousand yuan, you can chat with a cloned digital person of your relative on the electronic screen, who can speak, move, and have expressions, making him more realistic.

"AI resurrection" is a business with strong demand, and a market worth hundreds of millions of yuan is brewing. However, the merchants eyeing this pie are mixed. Some can make "talking photos" for sale by downloading software; some are digital human service providers, and in addition to selling digital humans such as live broadcasters, exhibitions, and hosts to the B-end, they also develop customized digital human APPs for the C-end; there are also a group of merchants who have identified the needs of users to resurrect their relatives and clone celebrities, and provide customized services to users by using self-developed + access to third-party technical interfaces.

"AI resurrection" may become a common service like taking photos in the future, but there are also issues such as data privacy and legal ethics. An industry insider said that the maturity of AI face-changing and voice-changing technology will also allow some people to commit fraud by taking advantage of users' longing for and trust in their loved ones. "Memories are good, but be careful not to be 'cut'," he reminded.

1. How much does it cost to “resurrect” a loved one?

Currently, the "AI resurrection" products on the market can be divided into three levels according to cost and technical difficulty, which also correspond to the three ways for users to "meet someone again" after AI "resurrects" someone.

The lowest level is photo-driven, commonly known as talking photo . Similar apps were popular a few years ago.

By using deep learning, image processing and other technologies, you can make the mouth and eyes of the people in the photo move; if you want the people in the photo to speak or sing, you need to use lip syncing (lip matching) and voice generation. "These technologies are mature and open source. After mass production, the single cost can be reduced to less than 10 yuan." Dong Huizhi, founder and president of Jialian Technology, who has 10 years of experience in AI entrepreneurship, said.

The second tier is pseudo live broadcast driven by expression capture, which replaces the image and voice of the deceased with a real-life model , conducts voice calls or video calls and other interactions, or generates short videos of blessings.

This uses voice cloning, AI face-changing, motion capture (expression capture), deepfake and other technologies to change people and voices. Dong Huizhi said that this is a particularly pleasing method, the technology is not new, and some AI scams use similar technology. The cost of motion capture equipment and labor is slightly higher, requiring several thousand yuan.

This type of "AI resurrection" video clip has a high number of views on short video platforms. Usually, the younger generation places an order for the elderly in the family, and asks someone to use the deceased's face and voice to talk to the elderly, and lie to the elderly that the deceased is working outside, continuing the "white lie". The elderly usually don't see anything unusual, but just wipe away tears frequently.

Mr. Sun used AI to "resurrect" his father

The third tier is the recently popular use of digital human technology to "resurrect" relatives . Because the products delivered are different, the costs are also different.

Generally speaking, the image and voice of a digital person are cloned by collecting photos, voice and other data of the person during his or her lifetime, and then a large language model is installed at the bottom layer to simulate the thoughts of the deceased, so that real-time text or voice communication can be carried out with the deceased.

51 Digital Human has this business, founder Chen Hong told "Dingjiao". Generally, the product delivered is a screen with a digital human, the large screen is as big as a TV, and the small screen can be the size of an iPad. After the user logs in to the account, he can see the digital image of the deceased relative, and can also interact with the digital human through voice or text. The product can also provide a voice call wake-up service.

"The customized 'resurrection of relatives' service usually costs more than 50,000 yuan." Chen Hong said that because customers pay for a long time, the details will be continuously optimized in the future. Some customers are a family that pools money together to do the work and have higher requirements.

"AI resurrection" has very high requirements for data quality. The more photos, videos or voice samples of the cloned person, and the clearer they are, the higher the similarity of the digital person. Some personal characteristic data, such as interests and hobbies, must be input to simulate their personality. "If the materials are not complete, the cost will increase further," said Chen Hong.

Therefore, the higher the accuracy and the more customized the data, the closer the effect is to real people, and the more expensive it is. The so-called accuracy is mainly reflected in the following aspects: the accuracy of the character's lip shape, the resolution clarity, the complexity of the action, the richness of the clothing and hairstyle and expression, the similarity of the voice (timbre, tone), whether there is electronic music, whether there is ups and downs (multiple emotions), whether there is interaction, whether the interaction is more in line with the person's personality, etc.

In the field of digital humans, the technical difficulty of perfectly cloning a person is "unlimited". Tang Xiaoou, the "resurrected" founder of SenseTime, is a case in point.

"Digital Human" Tang Xiaoou's speech at the annual meeting Source/Video screenshot

Luan Qing, general manager of the Digital Entertainment Division of the Digital Space Business Group of SenseTime Technology, told Dingjiao that different technologies were used to restore Mr. Tang's voice, appearance and smile. During the entire production process, SenseTime used its self-developed TTS voice generation model and intercepted four or five voices of Mr. Tang with different speaking styles as prompts. The total sound material was only a dozen seconds, but it restored Mr. Tang's northeastern accent, timbre, commonly used modal particles and intonation, and dry humor style.

In addition, SenseTime Ruying Digital Human Technology Team adopted Mr. Tang’s previous clear and effective videos, and used SenseTime’s self-developed video generation technology to generate actions and scene transitions, restoring actions such as walking, drinking water, and smiling expressions.

Due to the limited amount of materials, as well as the consideration of computing power and cost, many videos and products of digital people still look very "fake". "These are not complete 'resurrection' of digital people, and the upper limit of technology and service delivery cannot meet the minimum demand of people. " Chen Hong said.

If the image is not similar enough, it must be supplemented with emotional value. Add some interactive details to the design of the digital human. He gave an example, for example, a customer's grandfather likes the fourth child the most. During the chat, the grandfather suddenly said that the fourth child's birthday is next month, and the family should get together more often and the brothers should be harmonious. The customer will be moved at once.

Generally speaking, the first two tiers use relatively simple image processing, face-changing and voice-cloning technologies, which can see human faces and imitate voices, but the effects are crude due to low costs. The digital humans with a high degree of restoration and the ability to move and talk are all third-tier digital humans, and the more similar they are, the more expensive they are.

2. Who is making money from cloning humans?

From the past cases of "resurrecting relatives", we can see that most of them do not look like real people and make people feel out of place or even embarrassed. With the development of generative AI technology, Luan Qing observed that the technical feasibility and authenticity of "AI resurrection" have become higher, which can make people engaged and want to cry, further stimulating such needs.

As a result, an industrial chain came into being. In this industrial chain, some are followers, some are digital human service providers, some are AI practitioners, and some are players who specialize in customized AI resurrection (resurrection or cloning of relatives, celebrities, entrepreneurs, etc.).

On e-commerce platforms, many stores offer "AI resurrection" services, and the price of letting photos speak is mostly 10 yuan or 50 yuan. On short video platforms, many people also provide similar services in the name of "AI dreaming" and "AI healing." "The fee is cheap and the effect is rough, but this is a long-term long-tail market," said Dong Huizhi.

The "Let Photos Talk" service sold on Taobao

There are also gray areas in this type of business. Some short video bloggers have released videos of "resurrecting" deceased celebrities such as Coco Lee, Leslie Cheung, and Qiao Renliang, and let them sing and speak. Although they claim that "the purpose is to pay tribute and commemorate, without commercial purposes", such videos have received a lot of traffic. Some will also use this to divert traffic to the "relatives resurrection" business. At the same time, some stores that provide "AI resurrection" services also indicate "only for remembrance, please bypass facial recognition."

At present, technologically mature digital human service providers and AI practitioners tend to focus their business models on the B-end, such as AI customer service, digital human live broadcast, AI teachers, A hosts, AI medical care, etc. Some have launched similar tools for the C-end, but have not promoted them on a large scale.

Some netizens have used the voice big model of the big model startup MiniMax to clone the voice of a 90-second audio material, and used the Hailuowenwen APP under MiniMax to generate an intelligent body and have a voice conversation with it. Silicon Intelligence also has the "life cloning and digital immortality" business. Its Yan Emperor big model clones digital people based on the data provided by users, and users can interact with digital people in real time through the DUIX APP.

Silicon-based intelligent DUIX APP customizes digital life

Chen Hong and his team mainly focus on the high-precision customization market, with an average order starting at 50,000 yuan. They develop scenarios around large customers , such as digital cemeteries. When people go to the cemetery to pay tribute to their ancestors, the ancestors come out of the electronic screen to chat with everyone; for example, intelligent engineering of memorial halls, including the construction of architectural spaces; for example, the "resurrection" of celebrities, such as using early ancient paintings to "resurrect" Zhu Xi.

Zhu Xi made by 51 Digital Human

"For companies that provide customized services, not only the underlying technical capabilities are tested, but also the depth of the channels and the degree of implementation of the services , which determines whether users can truly use digital humans," Chen Hong believes.

From the perspective of a technology provider, Luan Qing believes that SenseTime Ruying's positioning is to empower various industries through digital human technology. Whoever understands the industry better can better serve users; whoever has a deeper solution can get more cake. "Resurrecting relatives is not a business that can be done purely from a technical perspective," said Luan Qing.

"The AI ​​revival business is more suitable for medium-sized teams," Dong Huizhi analyzed. Large companies have high operating and R&D costs. For the same set of technologies, they will give priority to B-side businesses that are standardized, mass-produced, and applicable to more scenarios. Chen Hong also said that large companies are unwilling to do it and small teams do not have the strength to do it. This market has an annual income of 5 million to 100 million, which is an opportunity for medium-sized teams."

At present, the threshold for “AI resurrection” seems to be low, but there are still many difficulties to be faced in order to do it well.

At present, the "AI resurrection" still has limitations such as technical limitations, lack of material reserves, and opposition from family members, making it still difficult to popularize.

The technical difficulty that most digital humans need to overcome is whether they can make it difficult to distinguish between humans and machines. Luan Qing mentioned that digital humans have made a step forward in terms of speech, movement, and scene connection, but in the long-term interaction process, they still cannot achieve true human-machine indistinguishability, and still need to be improved in terms of emotional communication, understanding, and consciousness.

If you really want to "resurrect" a person, details need to be reflected in all aspects. Chen Hong gave an example of an interactive scenario. When talking to the digital man grandfather, if the user asks information that is not in the digital man database, such as "Who is Nietzsche", the digital man will jump to the big model and answer according to the public answer. Although multiple rounds of conversations can be continued, it will not be realistic enough and the user's immersion will be interrupted.

3. “AI Resurrection” Still Needs Awe

There is a strong demand for "AI resurrection", but not everyone supports it, and the privacy, security and ethical issues hidden behind it cannot be ignored.

Supporters recognize the emotional value it provides. They believe that "resurrecting" relatives is a comfort to the living and a satisfaction for regrets, and is an example of technology being used for good.

Skeptics believe that people can never be "resurrected" and cannot be "authorized". Even if "AI resurrection" is authorized and approved by relatives, the wishes of the deceased person cannot be known.

"Digital immortality is not that easy. Even if a large model is used, what is ultimately replicated is just a GPT with the same face . When he talks to you with similar memory and abilities and IQ far superior to yours, will you definitely feel good?" Dong Huizhi asked in return.

On March 16, Qiao Renliang's father said that he could not accept the infringement of his son's portrait by the short video creator, and he felt uncomfortable and hoped that the other party would remove it as soon as possible. "They did not ask for our consent. My niece saw the video and sent it to me. This is opening up scars."

If there is a problem with the cloned digital human, it may cause secondary harm to the living. Too many film and television works have explored the ethical dilemma and the subtleties of human nature.

In one episode of Black Mirror, which aired in 2013, the heroine "resurrected" her husband who died in a car accident. Although she copied her husband's memory and body, she could not copy his emotions and choices. This AI husband does not need to sleep, will not get hurt, and will only act rigidly according to orders. The heroine realized that "you are not you, you are just a ripple", and finally locked the robot in the attic, but resented that she could not leave this false reality.

Screenshot of "Black Mirror" Source: Douban user Hiro

With the maturity of the "AI resurrection" industry chain, the demand and cases of "resurrecting relatives" and "cloning celebrities" have increased, and the many legal risks involved, such as privacy data leakage and AI fraud, have also attracted much attention.

In real life, cases of fraud through AI face-changing occur from time to time . This year's 315 Gala exposed many cases of fraud using AI technology to change faces and voices to impersonate relatives. In addition, fake celebrities are also a hard-hit area in scams. In March this year, Andy Lau's agency Yingyi Entertainment issued a statement on Weibo, saying that Andy Lau's voice was cloned and forged, reminding all parties to be vigilant against scams.

When replicating their loved ones, in order to achieve a higher degree of restoration, users can only be more open to the technology provider. This means that it is difficult for users to protect themselves. Once they encounter a scammer, it is difficult to determine whether it is a service or a scam.

In this regard, Chen Hong suggested that users who want to "resurrect their loved ones with AI" should sign a contract before placing an order, stipulating that the rights of the digital person belong to the individual and that the personal information provided will not be disclosed.

"The development of AI is accelerating, while safety issues, whether from the legal level, cultural level, civic awareness level or technical level, are lagging behind." Dong Huizhi said that ultimately the development of the industry still depends on the self-discipline of practitioners and the standardization of regulatory regulations.

Luan Qing said that industry norms and security are the major prerequisites for business development. Led by the China Academy of Information and Communications Technology, SenseTime has worked with a number of AI companies to develop the standards for "trusted digital humans".

There is no doubt that the relationship between humans and AI will become closer and more diverse in the future. The "AI resurrection" may become a standard service in the future, as simple as printing a photo.

With the advancement of technology, the concept of "AI resurrection" continues to upgrade. Compared with a video or a chat robot product, some people have proposed concepts such as digital immortality, digital companionship, and portable relatives. "Some wealthy people have started to copy themselves and build their own digital immortality library while they are still alive," said Dong Huizhi.

In 2015, Russian billionaire Dmitry Itskov launched an initiative to build a robot body for everyone to achieve immortality by 2045. This plan has faced some controversy. In 2022, Elon Musk mentioned on Twitter that he had uploaded his brain to the cloud and talked to a virtual version of himself, but some people thought that this was Musk promoting his brain-computer interface company Neuralink.

As the mystery of “AI resurrection” fades, the industry may also begin to enter an era of price involution. If the industry does not want to see the situation of “bad money driving out good money”, it needs to maintain awe.

Author: Su Qi Editor: Jin Yufan

Source: WeChat public account: Dingjiao (ID: dingjiaoone)