OpenAI

OpenAI loses its voice

The company hasn’t been the same since Sam Altman’s return — and its treatment of Scarlett Johansson should worry everyone

Casey Newton

May 20, 2024 — 11 min read

Scarlett Johansson on The Today Show last November. (Nathan Congleton / Getty Images)

OpenAI hasn't been the same company since Sam Altman returned as its CEO.

It’s a thought that has occurred to me repeatedly over the past week as the company has found mounting reasons to stop, reverse course, and apologize over product and personnel issues. And it’s one that feels increasingly urgent in a world where the company’s technology is poised to serve as the artificial intelligence backbone on both Microsoft and Apple devices.

On Monday, actress Scarlett Johansson revealed that the OpenAI CEO had been negotiating with her for the better part of a year to lend her voice to ChatGPT. “He told me that he felt that by my voicing the system, I could bridge the gap between tech companies and creatives and help consumers to feel comfortable with the seismic shift concerning humans and AI,” Johansson said in a statement published by NPR’s Bobby Allyn. “He said he felt that my voice would be comforting to people.”

Johansson declined to participate. But when OpenAI unveiled voices for ChatGPT in September, users were struck by the uncanny similarity between the voice it called Sky and Johansson’s. And it should not have been a surprise: Altman has said that Her, in which Johansson voices an AI assistant with whom the film’s protagonist falls in love, is his favorite movie. Last week, after OpenAI showcased an updated version of Sky that takes advantage of its latest models, Altman posted the movie’s title suggestively on X.

“When I heard the released demo, I was shocked, angered and in disbelief that Mr. Altman would pursue a voice that sounded so eerily similar to mine that my closest friends and news outlets could not tell the difference,” Johansson said.

But there is even more to the story, Johansson wrote.

“Two days before the ChatGPT 4.0 demo was released, Mr. Altman contacted my agent, asking me to reconsider,” she wrote. “Before we could connect, the system was out there.”

And so even as Altman attempted to renegotiate with Johansson behind the scenes, he intentionally drew attention to Sky’s similarity to Johansson in Her to his nearly 3 million Twitter followers — and released the product before finishing the negotiations.

In the moment, it looked like a grand success. The OpenAI team’s demonstration of a warmer, more emotional chatbot last week was one of the most impressive tech demos I’ve seen. In the end, I found its aggressive flirting unsettling, and seemingly at odds with Altman’s previous statements to me that the company would not pursue AI tools that doubled as romantic companions. But the demo unquestionably succeeded in captivating the tech world — and stealing thunder from Google on the eve of its annual developer conference.

Behind the scenes, though, Johansson was starting to ask questions. She hired attorneys, “who wrote two letters to Mr. Altman and OpenAI, setting out what they had done and asking them to detail the exact process by which they created the ‘Sky’ voice,” she wrote.

That did the trick: on Sunday night, OpenAI announced it would pull the Sky voice from ChatGPT. It also put up a blog post outlining the casting and recording process for the five voices it eventually selected for ChatGPT, Sky included.

OpenAI has taken pains to frame all of this as a big misunderstanding. CTO Mira Murati told reporters last week that Sky is not intended to sound like Johansson. And earlier on Monday, the company’s model behavior lead, Joanne Jang, told The Verge that “We’ve been in conversations with ScarJo’s team because there seems to be some confusion.”

OpenAI did not respond to my requests for comment today. But Johansson’s statement suggests that in reality, there was never any legitimate confusion about what was going on here. Altman asked to license her voice to realize the fantasy of a real-life Her; she declined; and the company proceeded anyway with a voice as similar to hers as it could get.

“I look forward to resolution in the form of transparency and the passage of appropriate legislation to help ensure that individual rights are protected,” Johansson said.

II.

Johansson is one of the world’s most famous actresses, and she speaks for an entire class of creatives who are now wrestling with the fact that automated systems have begun to erode the value of their work. OpenAI’s decision to usurp her voice for its own purposes will now get wide and justified attention.

At the same time, it’s possible that Johansson’s experience wasn’t even the most important thing that happened at OpenAI in the past week.

Last Tuesday, after all, Ilya Sutskever announced he was leaving the company. Sutskever, a co-founder of the company who is renowned both for his technical ability and his concerns about the potential of AI to do harm, was a leader of the faction that briefly deposed Altman in November. Sutskever reversed his position once it became clear that the vast majority of OpenAI employees were prepared to quit if Altman did not return as CEO. But Sutskever hasn’t worked for the company since.

In an anodyne public statement, Sutskever said only that he now plans to work on “a project that is very personally meaningful to” him.

Why not say more about the circumstances of his departure? Vox reported over the weekend that OpenAI’s nondisclosure agreements forbid criticizing the company for the duration of the former employee’s lifetime; forbids them from acknowledging the existence of the NDA; and forces them to give up all vested equity in the company if they refuse to sign it. (After the story drew wide attention, on Saturday Altman apologized and said the company would remove the provision about clawing back equity for violating the NDA.)

As OpenAI’s chief scientist, Sutskever also led its so-called “superalignment team,” which it formed last July to research ways of ensuring that advanced AI systems act safely and in accordance with human intent. Among other things, the company promised to dedicate 20 percent of its scarce and critical computing power to the project.

But Altman’s ouster led to wide disparagement of AI safety efforts, which were wrongly identified as the reason his co-workers wanted to fire him. (The actual-if-vague reason was that he was not “consistently candid in his communications,” according to the company’s former board, a charge that seems worth revisiting in light of Johansson’s experience with him.)

And so it perhaps should not be surprising that, upon his return, safety efforts were deprioritized. The superalignment team has been disbanded after less than a year, with its remaining employees absorbed into other teams. Jan Leike, who led the superalignment team under Sutskever, quit on Friday, saying OpenAI’s “safety culture and processes have taken a backseat to shiny products.”

“I joined because I thought OpenAI would be the best place in the world to do this research,” Leike wrote on X. “However, I have been disagreeing with OpenAI leadership about the company’s core priorities for quite some time, until we finally reached a breaking point.”

In response, Altman and OpenAI president Greg Brockman issued a long statement that gestured toward building things responsibly without addressing Laike’s criticism directly.

III.

What to make of all this?

On one hand, it’s not unusual for a fast-growing startup to go through some tumult as it scales up. On the other, OpenAI was not founded to be a typical fast-growing startup: it was founded to be a nonprofit research lab focused on the safe development of superhuman intelligence.

Since the release of ChatGPT — and much more so since Altman’s return from exile — OpenAI has looked like anything but. It has consistently pushed the boundaries of what AI can do, both from a technical standpoint and from the vantage of what it believes society can handle.

And last week, determined to show up Google on the eve of I/O, the company went through with its demo of a Johansson voice clone, against Johansson’s explicit wishes, because … Altman is a fan?

For the moment, the only cost to OpenAI from any of this has been a torrent of aggrieved social media posts. On Monday its APIs were at the center of a range of new PCs unveiled by its largest investor, Microsoft; next month they are expected to be announced as part of a big new partnership with Apple. One analysis last week suggested that the company’s mobile app revenue grew 22 percent the day it showcased the not-Johansson voice demo.

For years now, OpenAI told everyone that these were all secondary concerns — that its deeper ambition was something nobler, and more public-spirited. But since Altman’s return, the company has been telling a different story: a story about winning at all costs.

And why bother with superalignment, when there’s winning to do?

Why bother getting actresses’ permission, when the right numbers are all still going up?

Sponsored

Simplify your startup’s finances with Mercury

As a founder of a growing startup, you’re focused on innovating to attract customers, pinpointing signs of early product-market fit, and securing the funds necessary to grow. Navigating the financial complexities of a startup on top of it all can feel mystifying and incredibly overwhelming. More than that, investing time into becoming a finance expert doesn’t always promise the best ROI.

Mercury’s VP of Finance, Dan Kang, shares the seven areas of financial operations to focus on in The startup guide to simplifying financial workflows. It details how founders and early teams can master key aspects, from day-to-day operations like payroll to simple analytics for measuring business performance. Read the full article to learn the art of simplifying your financial operations from the start.

Read the Article

*Mercury is a financial technology company not a bank. Banking services provided by Choice Financial Group and Evolve Bank & Trust®; Members FDIC. Platformer has been a Mercury customer since 2020.