OpenAI’s superalignment meltdown: can any trust be salvaged?

Ilya Sutskever and Jan Leike from OpenAI‘s “superalignment” team resigned this week, casting a shadow over the company’s commitment to responsible AI development under CEO Sam Altman.

Leike, in particular, did not mince words. “Over the past years, safety culture and processes have taken a backseat to shiny products,” he declared in a parting shot, confirming the unease of those observing OpenAI‘s pursuit of advanced AI.

Yesterday was my last day as head of alignment, superalignment lead, and executive OpenAI?ref_src=twsrc%5Etfw”>@OpenAI.

— Jan Leike (@janleike) May 17, 2024

Sutskever and Leike are the latest entry in an ever-lengthening list of high-profile shake-ups at OpenAI.

Since November 2023, when Altman narrowly survived a boardroom coup attempt, at least five other key members of the superalignment team have either quit or been forced out:

• Daniel Kokotajlo, who joined OpenAI in 2022 hoping to steer the company toward responsible AGI development, quit in April 2024 after losing faith in leadership’s ability to “responsibly handle AGI.”
• Leopold Aschenbrenner and Pavel Izmailov, superalignment team members, were allegedly fired last month for “leaking” information, though OpenAI has provided no evidence of wrongdoing. Insiders speculate they were targeted for being Sutskever’s allies.
• Cullen O’Keefe, another safety researcher, departed in April.
• William Saunders resigned in February but is apparently bound by a non-disparagement agreement from discussing his reasons. 

Amid these developments, OpenAI has allegedly threatened to remove employees’ equity rights if they criticize the company or Altman himself, according to Vox. 

That’s made it tough to truly understand the issue at OpenAI, but evidence suggests that safety and alignment initiatives are failing, if they were ever sincere in the first place.

OpenAI’s controversial plot thickens

OpenA, founded in 2015 by Elon Musk and Sam Altman, was thoroughly committed to open-source research and responsible AI development.

However, as OpenAI’s vision has ballooned in recent years, the company has retreated behind closed doors. In 2019, it transitioned from a non-profit research lab to a “capped-profit” entity, fueling concerns about a shift toward commercialization over transparency.

Since then, OpenAI has guarded its research and models with iron-clad non-disclosure agreements and the threat of legal action against any employees who dare to speak out. 

Other key controversies in the startup’s short history include:

• In 2019, OpenAI stunned the AI ethics community by transitioning from a non-profit research lab to a “capped-profit” company, fueling concerns about a shift toward commercialization over transparency and the public good.
• Last year, reports emerged of closed-door meetings between Altman and world leaders like UK Prime Minister Rishi Sunak, in which the OpenAI CEO allegedly offered to share the company’s tech with British intelligence services, raising fears of an AI arms race. The company also formed deals with defense companies.
• Altman‘s erratic tweets have raised eyebrows, from musings about AI-powered global governance to admitting existential-level risk in a way that portrays himself as the pilot of a ship he cannot steer when that isn’t the case.
• In the most serious blow to Altman‘s leadership yet, Sutskever himself was part of a failed boardroom coup in November 2023 that sought to oust the CEO. While Altman managed to cling to power, it showed that Altman is well and truly bonded to OpenAI in a difficult way to pry apart. 

Examining this timeline, it’s challenging to distinguish OpenAI‘s controversies from its leadership.

The company is undoubtedly composed of talented individuals committed to contributing positively to society, but it falls under a company banner that Leike, Sutskever, and others have grown uncomfortable with.

OpenAI is becoming the antihero of generative AI

While armchair diagnosis and character assassination of Altman are irresponsible, his reported history of manipulation, lack of empathy for those urging caution, and pursuit of visions at the sacrifice of collaborators and public trust raise questions.

Conversations surrounding Altman and his company have become increasingly vicious across X, Reddit, and the Y Combinator forum.

For instance, there’s barely a shred of positivity on Altman’s recent response to Leike’s departure. That’s coming from people within the AI community, who perhaps have a stronger cause to empathize with Altman‘s position than most.

i’m super appreciative of @janleike‘s contributions to openai’s alignment research and safety culture, and very sad to see him leave. he’s right we have a lot more to do; we are committed to doing it. i’ll have a longer post in the next couple of days.

https://t.co/t2yexKtQEk

— Sam Altman (@sama) May 17, 2024

It’s become increasingly difficult to find Altman supporters within the community.

Well, what a shock. Jan and Ilya left OpenAI because they think I’m not prioritizing safety enough. How original.

Now I have to write some long, bs post about how much I care. But honestly, who needs safety when you can accelerate AI development at breakneck speeds and hope for… pic.twitter.com/BH45HgNDdR

— Sam Altman (Parody) (@SamAltsMan) May 17, 2024

While tech bosses are often polarizing, they usually win strong followings, as Elon Musk demonstrates among the more provocative types.

Others, like Microsoft CEO Satya Nadella, win respect for their corporate nouse and controlled, mature leadership style.

Let’s also mention how other AI startups, like Anthropic, manage to keep a fairly low profile despite their models equalling, even exceeding OpenAI‘s. OpenAI has created an intense, grandiose narrative that keeps it in the spotlight. 

In the end, we should say it how it is. The pattern of secrecy, the dismissal of concerns, and the relentless pursuit of headline-grabbing breakthroughs have all contributed to a sense that OpenAI is no longer a good-faith actor in AI. 

The moral licensing of the tech industry

Moral licensing has long plagued the tech industry, where the supposed nobility of the mission is used to justify all manner of ethical compromises. 

From Facebook’s “move fast and break things” mantra to Google’s “don’t be evil” slogan, tech giants have repeatedly invoked the language of progress and social good while engaging in questionable practices.

OpenAI’s mission to research and develop artificial general intelligence (AGI) “for the benefit of all humanity” invites perhaps the ultimate form of moral licensing.

Like Icarus, who ignored warnings and flew too close to the sun, Altman‘s laissez-faire attitude might propel the company beyond the limit of safety.

The danger is that if OpenAI does develop AGI, society might become tethered to its feet if it falls.

So, what can we do about it all? Well, talk is cheap. Robust governance, continuous progressive dialogue, and sustained pressure are key.

Some criticized the EU AI Act for being intrusive and destroying European competition, but maybe it’s right on the money. Maybe it’s better to create tight and intrusive AI regulations and back out as we better understand the technology’s trajectory.

As for OpenAI itself, as public pressure and media critique of OpenAI grow, Altman’s position could become less tenable. 

If he were to leave or be ousted, we’d have to hope that something positive fills the vacuum he’d leave behind. 

The post OpenAI’s superalignment meltdown: can any trust be salvaged? appeared first on DailyAI.