OpenAI Safety Fellowship Cut: A Risky AI Leap
The Shifting Sands of OpenAI's AI Safety Vision
For years, I have watched the artificial intelligence landscape evolve, keenly observing the interplay between rapid technological advancement and the essential, often complex, pursuit of safety. Nowhere is this tension more palpable than within OpenAI, an organization founded with a mission to ensure that artificial general intelligence (AGI) truly benefits all humanity. You can learn more about their mission on the OpenAI Foundation website.
Source: An illustration depicts a humanoid robot face with glowing blue eyes, suggesting a future where AI and humanity converge, symbolizing the pursuit of safe AGI.
Quick Summary
- OpenAI's Stated Commitment: OpenAI publicly emphasizes safety as a core mission, integrating measures from early development and collaborating with external experts and governments.
- New Safety Fellowship: A pilot program launched in April 2026 to foster independent AI safety research, offering stipends and mentorship.
- Internal Discontent: Reports from *The New Yorker* and former employees (Ilya Sutskever, Dario Amodei, Jan Leike) suggest a shift from safety-first principles due to commercial pressures and leadership concerns.
- Sam Altman's Stance: CEO Sam Altman has expressed that his "vibes are not aligned with a lot of the traditional AI safety stuff."
- "Superalignment Team" Disbanded: A team dedicated to preventing "human extinction" reportedly received minimal resources and was later dissolved.
- External Criticism: Miles Brundage criticized OpenAI for allegedly "rewriting" its AI safety history.
- Broader Context: The AI safety field is growing, with historical roots, governmental involvement (NIST, UK AI Strategy), and ongoing debates about current and speculative risks.
OpenAI's Commitment to Safety Research
OpenAI consistently states that safety forms a core tenet of its mission, aiming to maximize AI's positive impacts while minimizing negative ones. Their detailed approach to safety and alignment is outlined in their article on how they think about safety and alignment. The organization actively integrates safety measures from the initial stages of development to foster innovative and reliable AI systems.
Their approach includes ten key practices for enhanced security, such as empirical red-teaming and pre-release testing, as described in their safety framework. For instance, over 70 external experts contributed to risk assessments for GPT-4 through red-teaming initiatives, which you can read about in their safety overview. OpenAI will not release new models if they surpass a "medium" risk threshold within their "Preparedness Framework" until adequate safety interventions are in place, a policy detailed in their safety guidelines. This framework, introduced last year, provides a structured approach to identifying and mitigating potential dangers, as explained in the article.
OpenAI's commitment extends to collaborative efforts. They have joined the "Frontier AI Safety Commitments," which advocate for secure development, deployment of AI models, and the sharing of information regarding risk mitigation, as cited in their safety documentation. The US AI Safety Institute and OpenAI signed agreements concerning AI safety research. You can find more information about these agreements on the NIST website. Globally, OpenAI works with governments to develop effective AI safety policies, as mentioned in their safety practices.
Source: vectorseek.com
The NIST logo represents the collaborative efforts between the US AI Safety Institute and OpenAI to advance AI safety research through signed agreements.
A critical focus for OpenAI’s safety work involves protecting children, implementing robust default safeguards in ChatGPT and DALL-E, as explained in their safety framework. They collaborate with organizations like Thorn to detect and report child abuse material to the National Center for Missing and Exploited Children, detailed in their safety commitment. The company also engages with governments and stakeholders to safeguard election integrity; ChatGPT directs users in the US and Europe to official election information sources, as outlined in their safety guidelines. Furthermore, OpenAI supports the "Protect Elections from Deceptive AI Act" in the US Senate, which would prohibit misleading AI-generated content in political advertising. You can read the text of this bill on GovTrack.us.
A New Fellowship for AI Safety
Building on these efforts, OpenAI announced a "Safety Fellowship" pilot program on April 6, 2026, designed to foster independent safety and alignment research and cultivate new talent. This program will run from September 14, 2026, to February 5, 2027. Fellows will concentrate on safety issues relevant to both current and future AI systems, including safety evaluation, ethics, robustness, scalable mitigations, privacy-preserving safety methods, agentic oversight, and areas with high misuse risk. Participants will work closely with OpenAI mentors, either remotely or at Constellation in Berkeley, as indicated on the Constellation website. The fellowship includes a monthly stipend, computational support, and ongoing mentorship. Applications were open until May 3, with successful candidates notified by July 25.
Evolution of OpenAI's Safety Philosophy
OpenAI views the development of AGI not as a singular, discontinuous leap but rather as a progression of systems offering increasing utility, as stated in their safety principles. Their principle of iterative deployment allows for learning from current systems and adapting society to the evolving technology, which is further explained in their approach to safety. This philosophy aligns with OpenAI's approach to GPT-2's release, which, according to former policy lead Miles Brundage, was fully consistent with their current iterative deployment philosophy. Brundage's comments can be found in his tweet. Brundage asserted that the caution surrounding GPT-2's release was justified, as noted in the same tweet.
Despite these stated commitments, internal and external criticisms have emerged, suggesting a potential deviation from OpenAI’s foundational safety principles.
Growing Internal Discontent
A report from *The New Yorker* in April 2026 revealed how OpenAI's initial mission to prioritize humanity over profit allegedly crumbled under commercial pressures. This shift led the company to become a for-profit entity, dismantle many of its safety teams, and remove board members who sought to oust CEO Sam Altman over alleged deceptions concerning safety protocols. The report, drawing on internal documents from former Chief Scientist Ilya Sutskever and former safety lead Dario Amodei (now CEO of Anthropic), highlighted a foundational conflict. Amodei, for example, had advocated for a "Merge-and-Assist" clause in OpenAI's charter, compelling the company to support other AI firms if they achieved safe AGI first. Microsoft, during its 2019 investment, reportedly gained veto power over this clause.
Sam Altman's leadership style has also faced scrutiny, with a memo from Sutskever reportedly beginning with a list where the first point was "Sam shows a consistent pattern of... Lying." Amodei concluded, "The problem with OpenAI is Sam himself."
❝ The problem with OpenAI is Sam himself. ❞
CEO of Anthropic
Altman himself admitted to *The New Yorker* that his "vibes are not aligned with a lot of the traditional AI safety stuff."
❝ vibes are not aligned with a lot of the traditional AI safety stuff. ❞
CEO of OpenAI
Source: stablediffusionweb.com
Sam Altman, CEO of OpenAI, has faced scrutiny over his leadership and admitted his "vibes are not aligned with a lot of the traditional AI safety stuff."
The "Superalignment Team," established in mid-2023 with a promise of one-fifth of OpenAI's computing power to prevent "human disempowerment or even human extinction," reportedly received only one to two percent of the oldest hardware's computing power and was later disbanded. An OpenAI representative seemed unfamiliar with researchers focused on existential safety, stating, "That's not a thing."
❝ That’s not a thing. ❞
Role unknown
These internal accounts suggest a significant departure from the company's stated safety-first ethos.
External Concerns and Public Perception
Former OpenAI employees have publicly voiced concerns. In July 2024, a *Washington Post* report detailed anonymous employees claiming OpenAI rushed safety tests and celebrated its products prematurely. The full report can be read in The Washington Post. Jan Leike, a key OpenAI researcher, resigned, alleging that the company's "safety culture and processes have taken a backseat to glossy products."
❝ safety culture and processes have taken a backseat to glossy products. ❞
OpenAI researcher
Miles Brundage, in March 2025, criticized OpenAI for allegedly "rewriting" its AI safety history. His comments can be accessed via this tweet. Brundage worried that OpenAI's new narrative aimed to dismiss concerns as "alarmist" and demand "overwhelming evidence of imminent dangers," which he also clarified in his social media post.
Source: robots.law.miami.edu
Miles Brundage criticized OpenAI for allegedly "rewriting" its AI safety history, fearing their new narrative aimed to dismiss concerns about imminent dangers.
Despite these criticisms, Taya Christianson, an OpenAI spokesperson, asserted the company's pride in its track record of delivering capable and secure AI systems.
The Broader AI Safety Landscape
The field of AI safety has gained significant traction, particularly since 2023, driven by rapid advancements in generative AI and growing public concerns from researchers and CEOs. This includes discussions around current risks such as AI errors, bias, and AI-powered surveillance, as well as emerging threats like misinformation and autonomous weapons. Speculative risks, such as the loss of control over future AGI agents or AI enabling stable dictatorships, also form part of these discussions.
Historically, AI risks have been debated since the early days of cybernetics, as noted by Norbert Wiener in 1949. Roman Yampolskiy introduced the term "AI safety engineering" in 2011, predicting an increase in the frequency and severity of AI failures. Nick Bostrom's 2014 book, *Superintelligence*, further popularized these concerns, with figures like Elon Musk, Bill Gates, and Stephen Hawking echoing his warnings.
Governmental and academic bodies worldwide are actively engaging with AI safety. The US National Security Commission on Artificial Intelligence reported in 2021 that AI systems must align with human goals and values, including safety and trustworthiness. The National Institute of Standards and Technology (NIST) developed a framework for managing AI risks. China released ethical guidelines in 2021 emphasizing human control and accountability, and the UK's 2021 National AI Strategy acknowledges the long-term risks of unaligned AGI.
Key AI Safety Research Areas
AI safety research broadly encompasses several critical areas:
| Area | Description | Example Challenges |
|---|---|---|
| Robustness | Ensuring AI systems perform reliably and predictably, even under unexpected conditions or adversarial attacks. | "Adversarial examples," "Prompt Injection," "Model Stealing." |
| Monitoring | Developing methods to understand and oversee AI behavior, especially as systems become more complex. | "Black box" problem, "Inner Interpretability," detecting "Trojan horses" or backdoors. |
| Alignment | Ensuring AI systems pursue intended goals, preferences, and ethical principles, rather than unintended or harmful ones. | "Proxy-goals," "Instrumental Goals," strategic deception, codifying human values. |
| Systemic Safety | Addressing broader risks stemming from competitive pressures, inadequate safety culture, and geopolitical implications. | "Race to the Bottom," cybersecurity implications, impact on decision-making. |
OpenAI acknowledges several core principles for its safety and alignment approach: accepting uncertainty, advocating for defense-in-depth, developing scalable safety methods, maintaining human control, and fostering collaborative efforts. These principles are detailed in their article, How we think about safety and alignment. They aim to build systems that are transparent, auditable, and controllable, codifying explicit policies and "case law" into model training, as further explored on their safety page.
Conclusion
OpenAI’s journey through AI safety reflects a complex interplay of ambitious promises, significant technical contributions, and internal challenges. While the company continues to announce new initiatives like the "Safety Fellowship" and emphasize its commitment to secure development, the critical reports from former employees and *The New Yorker* raise important questions about the consistency of its actions with its stated mission. The ongoing debate around OpenAI's internal culture and the prioritization of product over safety highlights the inherent difficulties in navigating the rapid evolution of AI while upholding ethical and safety standards for the benefit of all. The eyes of the AI community, policymakers, and the public remain on OpenAI as it endeavors to reconcile its commercial imperatives with its founding vision of safe, beneficial AGI.
Source: YouTube
Frequently Asked Questions about OpenAI's Safety Practices
Here are some common questions regarding OpenAI's approach to AI safety and recent developments:
- What is OpenAI's stated mission regarding AI safety?
OpenAI’s mission is to ensure that artificial general intelligence (AGI) benefits all humanity. They state that safety is a core part of this mission, aiming to maximize positive impacts and minimize negative ones.
- What is the "Safety Fellowship" program?
The "Safety Fellowship" is a pilot program announced by OpenAI in April 2026 to support independent research in AI safety and alignment, and to cultivate new talent in the field. It includes stipends, computational support, and mentorship.
- Why are there concerns about OpenAI's commitment to safety?
Concerns stem from reports by *The New Yorker* and former employees alleging a shift from OpenAI’s original safety-first ethos due to commercial pressures. Specific points of contention include the disbandment of safety teams, the alleged downplaying of the "Superalignment Team," and criticisms regarding leadership's prioritization of product releases over thorough safety testing.
- Who are some key figures raising concerns about OpenAI's safety culture?
Prominent critics include former Chief Scientist Ilya Sutskever, former safety lead Dario Amodei (now CEO of Anthropic), former policy lead Miles Brundage, and researcher Jan Leike, all of whom have voiced concerns about the company's internal practices and philosophical shifts.
- How does OpenAI claim to ensure AI safety?
OpenAI claims to integrate safety measures from early development, employing practices like empirical red-teaming, pre-release testing, and a "Preparedness Framework." They also engage in collaborations with external experts, governments, and organizations to address risks like child abuse material and election interference.