Media & Entertainment

The team behind Baidu’s first smart speaker is now using AI to make films

Comment

rct studio
Image Credits: rct studio

The HBO sci-fi blockbuster Westworld has been an inspiring look into what humanlike robots can do for us in the meatspace. While current technologies are not quite advanced enough to make Westworld a reality, startups are attempting to replicate the sort of human-robot interaction it presents in virtual space.

Rct studio, which just graduated from Y Combinator and ranked among TechCrunch’s nine favorite picks from the batch, is one of them. The “Westworld” in the TV series, a far-future theme park staffed by highly convincing androids, lets visitors live out their heroic and sadistic fantasies free of consequences.

There are a few reasons why rct studio, which is keeping mum about the meaning of its deliberately lower-cased name for later revelation, is going for the computer-generated world. Besides the technical challenge, playing a fictional universe out virtually does away the geographic constraint. The Westworld experience, in contrast, happens within a confined, meticulously built park.

“Westworld is built in a physical world. I think in this age and time, that’s not what we want to get into,” Xinjie Ma, who heads up marketing for rct, told TechCrunch. “Doing it in the physical environment is too hard, but we can build a virtual world that’s completely under control.”

rct studio
Rct studio wants to build the Westworld experience in virtual worlds. / Image: rct studio

The startup appears suitable to undertake the task. The eight-people team is led by Cheng Lyu, the 29-year-old entrepreneur who goes by Jesse and helped Baidu build up its smart speaker unit from scratch after the Chinese search giant acquired his voice startup Raven in 2017. Along with several of Raven’s core members, Lyu left Baidu in 2018 to start rct.

“We appreciate a lot the support and opportunities given by Baidu and during the years we have grown up dramatically,” said Ma, who previously oversaw marketing at Raven.

Let AI write the script

Immersive films, or games, depending on how one wants to classify the emerging field, are already available with pre-written scripts for users to pick from. Rct wants to take the experience to the next level by recruiting artificial intelligence for screenwriting.

At the center of the project is the company’s proprietary engine, Morpheus. Rct feeds it mountains of data based on human-written storylines so the characters it powers know how to adapt to situations in real time. When the codes are sophisticated enough, rct hopes the engine can self-learn and formulate its own ideas.

“It takes an enormous amount of time and effort for humans to come up with a story logic. With machines, we can quickly produce an infinite number of narrative choices,” said Ma.

To venture through rct’s immersive worlds, users wear a virtual reality headset and control their simulated self via voice. The choice of audio came as a natural step given the team’s experience with natural language processing, but the startup also welcomes the chance to develop new devices for more lifelike journeys.

“It’s sort of like how the film Ready Player One built its own gadgets for the virtual world. Or Apple, which designs its own devices to carry out superior software experience,” explained Ma.

On the creative front, rct believes Morpheus could be a productivity tool for filmmakers as it can take a story arc and dissect it into a decision-making tree within seconds. The engine can also render text to 3D images, so when a filmmaker inputs the text “the man throws the cup to the desk behind the sofa,” the computer can instantly produce the corresponding animation.

Path to monetization

Investors are buying into rct’s offering. The startup is about to close its Series A funding round just months after banking seed money from Y Combinator and Chinese venture capital firm Skysaga, it told TechCrunch.

The company has a few imminent tasks before achieving its Westworld dream. For one, it needs a lot of technical talent to train Morpheus with screenplay data. No one on the team had experience in filmmaking, so it’s on the lookout for a creative head who appreciates AI’s application in films.

rct studio
Rct studio’s software takes a story arc and dissects it into a decision-making tree within seconds. / Image: rct studio

“Not all filmmakers we approach like what we do, which is understandable because it’s a very mature industry, while others get excited about tech’s possibility,” said Ma.

The startup’s entry into the fictional world was less about a passion for films than an imperative to shake up a traditional space with AI. Smart speakers were its first foray, but making changes to tangible objects that people are already accustomed to proved challenging. There has been some interest in voice-controlled speakers, but they are far from achieving ubiquity. Then movies crossed the team’s mind.

“There are two main routes to make use of AI. One is to target a vertical sector, like cars and speakers, but these things have physical constraints. The other application, like Alpha Go, largely exists in the lab. We wanted something that’s both free of physical limitation and holds commercial potential.”

The Beijing and Los Angeles-based startup isn’t content with just making the software. Eventually, it wants to release its own films. The company has inked a long-term partnership with Future Affairs Administration, a Chinese sci-fi publisher representing about 200 writers, including the Hugo award-winning Cixin Liu. The pair is expected to start co-producing interactive films within a year.

Rct’s path is reminiscent of a giant that precedes it: Pixar Animation Studios. The Chinese company didn’t exactly look to the California-based studio for inspiration, but the analog was a useful shortcut to pitch to investors.

“A confident company doesn’t really draw parallels with others, but we do share similarities to Pixar, which also started as a tech company, publishes its own films, and has built its own engine,” said Ma. “A lot of studios are asking how much we price our engine at, but we are targeting the consumer market. Making our own films carry so many more possibilities than simply selling a piece of software.”

Our 9 favorite startups from Y Combinator W19 Demo Day 2

More TechCrunch

After two years of preparation and four delays over the past several months due to technical glitches, Indian space startup Agnikul has successfully launched its first sub-orbital test vehicle, powered…

India’s Agnikul launches 3D-printed rocket in sub-orbital test after initial delays

Struggling EV startup Fisker has laid off hundreds of employees in a bid to stay alive, as it continues to search for funding, a buyout or prepare for bankruptcy. Workers…

Fisker cuts hundreds of workers in bid to keep EV startup alive

Chinese EV manufacturers face a new challenge in their pursuit of U.S. customers: a new House bill that would limit or ban the introduction of their connected vehicles. The bill,…

Chinese EV makers, and their connected vehicles, targeted by new House bill

With the release of iOS 18 later this year, Apple may again borrow ideas third-party apps. This time it’s Arc that could be among those affected.

Is Apple planning to ‘sherlock’ Arc?

TechCrunch Disrupt 2024 will be in San Francisco on October 28–30, and we’re already excited! This is the startup world’s main event, and it’s where you’ll find the knowledge, tools…

Meet Visa, Mercury, Artisan, Golub Capital and more at TC Disrupt 2024

Featured Article

The women in AI making a difference

As a part of a multi-part series, TechCrunch is highlighting women innovators — from academics to policymakers —in the field of AI.

9 hours ago
The women in AI making a difference

Cadillac may seem a bit too traditional to hang its driving cap on EVs. And yet, that hasn’t stopped the GM brand from rolling out — or at least showing…

The Cadillac Optiq EV starts at $54,000 and is designed to hook young hipsters

Ifeel is being offered as part of an employer’s or insurance provider’s healthcare coverage.

Mental health insurance platform ifeel raises a $20 million Series B

Instead of opening the user’s actual browser or a WebView, Custom Tabs let users remain in their app while browsing.

Google Chrome becomes a ‘picture-in-picture’ app

Sanil Chawla remembers the meetings he had with countless artists in college. Those creatives were looking for one thing: sustainable economic infrastructure that could help them scale rather than drown…

Slingshot raises $2.2 million to provide financial services to artists

A startup called Firefly that’s tackling the thorny and growing issue of cloud asset management with an “infrastructure as code” solution has raised $23 million in funding. That comes on…

Firefly forges on after co-founder murdered by Hamas

Mistral, the French AI startup backed by Microsoft and valued at $6 billion, has released its first generative AI model for coding, dubbed Codestral. Like other code-generating models, Codestral is…

Mistral releases Codestral, its first generative AI model for code

Pinterest announced today that it is evolving its Creator Inclusion Fund to now be called the Pinterest Inclusion Fund. Pinterest teamed up with Shopify’s Build Black and Build Native programs…

Pinterest expands its Creator Fund to allow founders

Alex Taub, a longtime founder with multiple exits under his belt, believes it’s time to disrupt the meme industry. “I have this big thesis that meme tech is going to…

This founder says meme tech is the next big thing

Lux, the startup behind popular pro photography app Halide and others, is venturing into video with its latest app launch. On Wednesday, the company announced Kino, a new video capture app…

Kino is a new iPhone app for videographers from the makers of Halide

DevOps startup Harness has shown itself to be an ambitious company, building a broad platform of services while also dabbling in M&A when it made sense to fill in functionality.…

Harness snags Split.io as it goes all in on feature flags and experiments

Microsoft’s Copilot, a generative AI-powered tool that can generate text as well as answer specific questions, is now available as an in-app chatbot on Telegram, the instant messaging app.  Currently…

Microsoft’s Copilot is now on Telegram

HBO’s new documentary, “MoviePass, MovieCrash,” tells a story that many of us know about: how MoviePass, the subscription-based movie ticketing startup, was a catastrophic failure. After a series of mishaps…

MoviePass co-founders speak their truth in HBO’s new documentary 

The watch features a variety of different 3D games, unlocking more play time the more kids move.

Fitbit’s new kid smartwatch is a little Wiimote, a little Tamagotchi

In the video, a crowd is roaring at a packed summer music festival. As a beat starts playing over the speakers, the performer finally walks onstage: It’s the Joker. Clad…

Discord has become an unlikely center for the generative AI boom

After the Wirecard scandal, Germany’s financial regulator BaFin started to look more closely at young fintech startups that wanted to grow at a rapid pace — it’s better to be…

Germany’s financial regulator ends anti-money laundering cap on N26 signups after $10M fine

Among other things, this includes the ability to trace code from source to binary packages across both platforms, single sign-on support and unified project structures.

JFrog and GitHub team up to closely integrate their source code and binary platforms

The company’s public fund disbursement and e-commerce platform makes accepting school tuition and enabling educational enrichment more accessible. 

Tech startup Odyssey goes on journey to help states implement school choice programs

A new startup called Kinnect aims to help people privately save generational memories, traditions, recipes and more. The company’s app, launched this month, lets people create invite-only spaces where they…

Kinnect’s new app aims to help families record and store generational memories

Spotify has hiked its premium subscription in France by an eye-watering €0.13, in response to a new music-streaming tax.

Spotify hikes subscription price in France by 1.2% to match new music-streaming tax

The European Union has taken the wraps off the structure of the new AI Office, the ecosystem-building and oversight body that’s being established under the bloc’s AI Act. The risk-based…

With the EU AI Act incoming this summer, the bloc lays out its plan for AI governance

Solutions by Text, a company that gives people a way to pay their bills and apply for loans via text messaging, has secured $110 million in new growth funding. Edison…

Bootstrapped for over a decade, this Dallas company just secured $110M to help people pay bills by text

Owners of small- and medium-sized businesses check their bank balances daily to make financial decisions. But it’s entrepreneur Yoseph West’s assertion that there’s typically information and functions missing from bank…

Relay raises $32.2 million to help smaller businesses manage their cash flow

When other firms were investing and raising eye-popping sums, Clean Energy Ventures took a different approach. It appears to be paying off.

How Clean Energy Ventures avoided the pandemic bubble and raised a $305M fund

PwC, the management consulting giant, will become OpenAI’s biggest customer to date, covering 100,000 users.

OpenAI signs 100K PwC workers to ChatGPT’s enterprise tier as PwC becomes its first resale partner