Sony Showcases AI-Driven Aloy Prototype in Leaked Footage

A leaked internal video has exposed Sony's exploration of AI-driven PlayStation characters.

The Verge reported on a leaked video, allegedly produced by PlayStation Studios' advanced technology group, featuring Aloy from the Horizon series to demonstrate AI-enhanced game characters.

The video was removed from YouTube after a copyright claim by Muso, an internet enforcement firm reportedly associated with Sony Interactive Entertainment, indicating its authenticity. IGN has reached out to Sony for comment.

According to The Verge, the video shows Sony Interactive Entertainment’s software engineering director, Sharwin Raghoebardajal, engaging in a dialogue with an AI-driven Aloy using voice prompts, powered by AI-generated speech and facial animations.

The speech recognition uses OpenAI’s Whisper for speech-to-text, conversational AI leverages GPT-4 and Llama 3 for dialogue and decision-making, speech synthesis employs Sony’s Emotional Voice Synthesis (EVS) system, and facial animations utilize Sony’s Mockingbird technology.

In the demo, Raghoebardajal asks Aloy about her well-being, and she responds: "Hi, I’m doing okay, just battling a bit of a sore throat. How about you?"

The voice from Aloy is not that of Ashly Burch, the character’s video game voice actor, but a robotic tone akin to common text-to-speech generators used for social media narration. AI Aloy’s facial movements appear rigid, and her eyes lack vitality during the conversation.

When asked about her search for her mother, Aloy says: "I learned I’m a clone of Dr. Elizabeth Sobeck, which helped me understand my purpose and origins."

On being a clone, she responds: "Being a clone feels distinctive, tying me to the past while letting me forge my own path forward."

The demo shifts to the Horizon Forbidden West game world, where Raghoebardajal continues the conversation while playing. It’s striking to see a player interact with a character they control, especially given the irony of using Aloy, considering the Horizon series’ narrative.

What’s the Top PlayStation 5 Game?

Choose a winner

The prototype, developed with Horizon studio Guerrilla Games, was created to showcase the technology internally at Sony. “This is merely a preview of what’s possible,” Raghoebardajal notes in the video. While it highlights Sony’s investment in AI-driven PlayStation characters, there’s no confirmation of integrating this tech into public-facing PlayStation products, and it’s unclear if it’s feasible for PS5 games at this stage.

Sony’s behind-the-scenes work aligns with industry trends, as competitors like Microsoft, which recently unveiled an AI called Muse for game design ideation, are also heavily investing in AI technology.

Generative AI remains a polarizing topic in the video game and entertainment sectors, both of which have faced significant layoffs recently. Criticism of generative AI stems from ethical concerns, rights issues, and its challenges in creating content that resonates with audiences. For example, Keywords Studios’ attempt to develop a game entirely with AI failed, with the company noting to investors that AI “couldn’t replace talent.”

In September, EA stated that AI was “central” to its operations, while Capcom revealed it was testing generative AI to generate the “countless ideas” needed for in-game environments.

Asad Qizilbash, head of PlayStation Productions and product at PlayStation Studios, emphasized AI’s role in appealing to Gen Z and Gen Alpha gamers, who value “personalization in everything.”

“For example, non-player characters could respond to players’ actions, creating a more tailored experience,” Qizilbash explained. “This matters to Gen Z and Gen Alpha, the first digitally native generations who seek personalized, meaningful experiences.”

Activision recently acknowledged using generative AI for certain Call of Duty: Black Ops 6 assets, sparking backlash over an “AI-generated” zombie Santa loading screen.