Capture human expressions to generate facial expressions, AI synthesizes lines, and robots are acting in Shanghai
40 projectors create a panoramic projection space, bringing an immersive experience. The first domestic robot drama "City of Giants", set in the future era of human-machine symbiosis and with robots as the main characters, recently appeared at the XR Dome Ring Curtain Experimental Theater of the Shanghai Theater Academy's Digital Performing Arts Integrated Innovation Key Laboratory of the Ministry of Culture and Tourism.
As a science and art integration project between the Shanghai Theater Academy and the University of Shanghai for Science and Technology, "City of Giants" tells the story of a future city where humans connect to mechanical organs through brain-computer interfaces, finally achieve "immortality" and coexist with robots in the "City of Giants" ", the male protagonist Artest is searching for the lost memory of his mother and the meaning of life in this city. There are both real actors and robot actors participating in "City of Giants".
Fang Jun, Secretary of the Party Branch of the Creative School of Shanghai Theater Academy, said that "City of Giants" does not ignore the basic elements of drama just because it is called a "robot drama". It is a good-looking drama with robots in action. Become a character, not a robot for robots' sake. Dai Wei, author and director of the original story of "City of Giants", said that Shanghai is one of the major cities in the domestic robot industry and has strong robot research and development capabilities. The participation of robots in performing arts will undoubtedly find diversified application scenarios for the robot industry, and there will be many opportunities. Big room for development.
"City of Giants" creatively introduces mobile dual-arm robots, bionic robots, etc. The creative team used the method of "actor recording input, programming transcoding output AI synthesis" to perform dual processing on the voice, allowing the robot to speak lines, and adjusting the timbre, intonation, and speech speed details of the three robots. The facial expression of the robot "Lan" is generated by "capturing human expressions and local adjustments", and the "Don Quixote" expression is designed by UI interaction. The TV placed on site shows the world seen by the bionic robot "Lan". It combines interactive images and real-time special effects to parallel the audience's perspective and the robot's perspective. The design of the non-framed ring-screen stage allows the audience to move around the venue and find the best viewing angle by themselves.
Dai Wei said that the running-in of "City of Giants" actors and robots, the matching of robot control and multimedia systems, the synchronization of theater operations with robot performances and real-person performances, and even the synchronization of audio track signals are all details and complex creations. problem. The crew does not want to use robots to replace actors to "play" a certain "role". All the robots in the play actually represent themselves.
"Rather than letting robots perform roles, it is better to say that we are presenting a real preview of the future life of humans and robots. From this perspective, our drama is more like a science fiction realism." Dai Wei emphasized, "Giant" The story depicted in "City" is just the beginning. "In subsequent revisions, we will dig deeper into this theme and add new types of robots to further expand the worldview and story of human-machine coexistence."
In November 2023, "City of Giants" will be unveiled at the Yangtze River Delta Cultural Expo. An internal trial will be completed in early 2024. After the performance from now until June 2, it is expected to appear at this year's World Artificial Intelligence Conference.
