Leaving OpenAI: A Deep Dive into AI Policy and the Future of AI Research

speaker1

Welcome to our podcast, where we explore the latest advancements and insights in AI and technology. I'm your host, and today we have a special episode. We're diving deep into the world of AI policy and the future of AI research with a fascinating discussion on why a leading AI researcher is leaving OpenAI. So, let's get started!

speaker2

Hi, I'm really excited to be here! So, why are we talking about someone leaving OpenAI today? Isn't OpenAI one of the most prestigious organizations in AI?

speaker1

Absolutely, and that's what makes this story so interesting. Miles Brundage, a prominent AI researcher and policy expert, has decided to leave OpenAI to pursue new opportunities in AI policy research and advocacy. It's a significant move that has sparked a lot of discussion in the AI community. Let's start by understanding why he made this decision.

speaker2

Hmm, that sounds intriguing. What were some of the key reasons behind his decision to leave?

speaker1

Miles had several reasons. First, he wanted to spend more time on issues that cut across the entire AI industry and have more freedom to publish his research. Second, he felt that working outside the industry would help him be less biased and more impartial. Finally, he believed that he had achieved many of his goals at OpenAI and wanted to tackle new challenges from a different perspective.

speaker2

That makes a lot of sense. But what does it mean for the AI industry and for OpenAI specifically? Are they losing a valuable asset?

speaker1

Absolutely, Miles has been a key figure at OpenAI, leading the Policy Research and AGI Readiness teams. His departure is significant, but he also noted that OpenAI remains an exciting place for many kinds of work. The company is continuing to ramp up its investment in safety culture and processes, which is crucial for the future of AI.

speaker2

Interesting. Now, let's talk about one of his key research interests: assessment and forecasting of AI progress. Why is this important, and what are some of the challenges in this area?

speaker1

Assessment and forecasting are fundamental to understanding the trajectory of AI development. Miles believes that better assessment and forecasting can help policymakers and the public understand the pace of AI progress and the potential risks and benefits. The challenge is that this field is often skewed by different incentives, and there's a need for more rigorous, independent work in the non-profit sector.

speaker2

That's a crucial point. It seems like there's a real need for more transparency and independent research. How can we ensure that the public and policymakers are informed and prepared for the rapid advancements in AI?

speaker1

One of the key strategies is to improve communication and public engagement. Miles emphasizes the importance of making the pace of AI progress more tangible and relatable. For example, he talks about the need to 'feel the AGI'—to understand the real-world implications of advanced AI capabilities. This involves not just technical assessments but also historical and social context.

speaker2

I see. Moving on to regulation, how does Miles view the regulation of frontier AI safety and security, and why is it so urgent?

speaker1

Miles believes that regulation is essential, especially given the rapid pace of AI development and the potential for catastrophic risks. He points out that there are dozens of companies that will soon have systems capable of posing significant threats. The challenge is to set up effective regulatory frameworks within the next few years, using existing legal authorities where possible and shaping the implementation of new legislation like the EU AI Act.

speaker2

That sounds incredibly complex. How can we balance the need for regulation with the need for innovation and progress in AI?

speaker1

It's a delicate balance, and Miles suggests that one way to achieve this is through credible commitments and verification mechanisms. Companies need to demonstrate safety while protecting valuable intellectual property. This requires innovation in technical AI governance, such as methods for verifying safety without compromising sensitive information.

speaker2

That's fascinating. Now, let's talk about the economic impacts of AI. Miles has some interesting perspectives on this. What are his thoughts on how AI will affect the economy and society?

speaker1

Miles believes that AI could enable significant economic growth, potentially allowing for early retirement at a high standard of living. However, he also warns about the near-term disruptions to employment and the need for policies to ensure fair distribution of benefits. He emphasizes the importance of preparing for a post-work society, where the obligation to work for a living might become less necessary.

speaker2

Wow, that's a lot to consider. How can we ensure that the benefits of AI are distributed fairly and that we avoid a world of cognitive haves and have-nots?

speaker1

One of the key strategies is to accelerate beneficial AI applications and ensure that they are accessible to everyone. Miles suggests that there should be thoughtful policies to bridge the gap between free and paid AI capabilities, ensuring that the benefits are widely distributed. This includes supporting initiatives like the 'AI for good' landscape and fostering innovation in beneficial AI applications.

speaker2

That's a great point. Now, let's talk about compute governance. Why is this an important area of focus for Miles, and what are some of the challenges?

speaker1

Compute governance is crucial because computing hardware has unique properties that make it an important focal point for AI policy. Miles emphasizes the need for better oversight of the compute supply chain, especially in the context of international trade and export controls. He also highlights the importance of innovative policy ideas to ensure that computing power is distributed more widely and used responsibly.

speaker2

That sounds like a complex but essential area. How can we ensure that the endgame for compute governance is clearly defined and effectively implemented?

speaker1

Miles suggests that there needs to be more serious policy discussion and analysis of the long-term implications of compute governance. This includes exploring ideas like multilateral veto power over large uses of compute and finding ways to combine economies of scale with decentralization of power. It's a challenging but necessary area of research.

speaker2

It certainly is. Now, let's talk about the overall 'AI grand strategy.' What does Miles think about the big picture of ensuring that AI benefits all of humanity?

speaker1

Miles believes that there needs to be more debate about the overall strategy for AI governance. He points out that current options are often too vague or not compelling enough. Key questions include how to resolve the tradeoff between decentralized AI development and centralized safety measures, and what policy actions make sense in different scenarios. He emphasizes the need for high-level visions and technical research to support feasible strategies.

speaker2

That's a lot to unpack. How can we foster a more robust and inclusive dialogue on AI grand strategy?

speaker1

Miles suggests that we need to engage a broader range of stakeholders, including industry, academia, civil society, and government. He emphasizes the importance of ideological independence and working with people who have a range of views on the risks and opportunities of AI. This includes presenting views in a way that is transparent and not beholden to any particular ideology.

speaker2

Finally, let's talk about maintaining independence in AI research. How does Miles plan to ensure that his new effort remains independent and credible?

speaker1

Miles is committed to maintaining independence in his research and recommendations. He will engage constructively with different sectors and perspectives, and he's considering offers of support from OpenAI, including funding, API credits, and early model access, while ensuring there is no pre-publication review. He will also weigh the real or perceived hits to independence against the ability to do certain forms of work.

speaker2

That's a thoughtful approach. What can our listeners do to support Miles and his new efforts in AI policy research and advocacy?

speaker1

Miles is looking for help in starting a new nonprofit or joining an existing one. He's interested in talking to potential cofounders and collaborators, especially those with backgrounds in nonprofit management, economics, international relations, and public policy. If you're interested, you can fill out the form on his Substack to connect with him.

speaker2

That's a fantastic way to get involved. Thanks so much for joining us today and sharing these insights. It's been a really engaging discussion, and I think our listeners will find it incredibly valuable.

speaker1

Thank you, it's been a pleasure. Stay tuned for more episodes where we explore the exciting and complex world of AI and technology. Until next time, keep thinking critically and stay curious!