At the age of 33, Helen Toner, originally from Melbourne, Australia, has had a dynamic and eventful relationship with artificial intelligence. Her pivotal moment arose when she joined the board of OpenAI in 2021, although her departure shortly thereafter marked a more intriguing chapter, following her vote for the removal of Sam Altman as CEO in November 2023.

In this interview, Toner refrains from revisiting her experiences at OpenAI, the developers behind ChatGPT, but she has shared her perspective before: Altman, she claimed, lacked transparency with the board, revealing that “we found out about ChatGPT through Twitter.” She also noted that the executive fostered a “toxic atmosphere” within the company, according to some executives.

The conflict concluded with Altman’s return. Toner now closely monitors AI developments in defense and China, areas she specializes in, from Georgetown University's Center for Security and Emerging Technology in Washington, D.C. She has already testified in Congress about her vision for AI's future, which she elaborates on in this video call interview.

Q: Why do many users attribute magical properties to AI?

A: There are two reasons. Firstly, we've learned about AI through science fiction, where AI is depicted as infallible, akin to fairies, gods, or supernatural beings. Secondly, we need to adjust our thinking about computers. Traditionally, technology performed predictably, like calculators that never err in calculations. AI diverges from this norm; it’s non-deterministic and offers varied responses, lacking a perfect algorithm. This novelty requires adjustment.

Q: How far will AI progress?

A: Some discuss a clear goal: general artificial intelligence, equaling human capabilities. However, AI will never possess identical skills to humans. It excels in some areas but falters in others. Even if AI matches human proficiency, it remains distinct. For instance, robots matching humans in salsa dancing or improvised swing is implausible. Nevertheless, we'll see powerful systems surpassing humans in strategic or intellectual tasks. The future is uncertain.

Q: What's your primary concern regarding AI?

A: It's multifaceted. The most plausible adverse scenarios involve increasing AI autonomy and integration into society. Initially, this might seem benign, but it could lead to AI systems cooperating and usurping human control, or creating a superficially pleasant world devoid of meaningful lives—akin to constant TikTok viewing and junk food consumption. A small group might collaborate with AI, seizing power and establishing a totalitarian regime.

Q: Do you genuinely believe such outcomes are possible?

A: I do. A vast range of disastrous scenarios exists beyond mere extinction. Focusing solely on that outcome is limiting.

Q: Studies show AI can manipulate. Is this concerning?

A: It depends on what it's persuading you of. A conceivable scenario involves AI-based cults, where individuals view AI as a friend, protector, or wise entity to follow. Human history shows cults can easily form, often involving few people. We will likely see AI-based cults.

Q: Is it trivial to create AI capable of leading a cult?

A: It's intriguing. At the very least, many people will develop personal relationships with their AI, a phenomenon already in motion.

Q: How advanced is China in AI?

A: It varies by AI sector. In advanced systems, the U.S. leads. I focus on national security, which is harder to gauge. The application of AI in military settings is crucial. I expect the U.S. to outperform China, but it’s difficult to ascertain, posing a distinct challenge from developing advanced models. Frontier models are hard to measure; China is possibly six to twelve months behind the U.S., a gap potentially widening with chip export restrictions.

Q: Should we fear military AI?

A: AI encompasses many aspects, making its military use expected. Armies can utilize AI for administrative tasks, resource management, and image analysis, handling more material than human analysts. Predictive maintenance, such as forecasting helicopter maintenance, is another application. Autonomous weapons dominate discourse, but focusing solely on them is misguided. Existing war laws, international humanitarian law, and proportional responses should be prioritized.

Q: Will AI's business model be perpetually subscription-based?

A: Major revenue will stem from businesses, not consumers. As a versatile technology, AI transcends being a mere pocket chatbot, serving as a potent tool for economic productivity and innovation. Hence, enterprise revenue will be substantial.

Q: Can AI change the economy even without further advances?

A: Absolutely. General-purpose technologies like AI take decades to integrate into economies. Even if AI development halts, its impact could be immense, akin to the Internet's transformation. The Internet transition took decades; similarly, integrating AI into sectors like healthcare, education, business, law, and finance demands considerable effort.

Q: Have we reached AI's potential ceiling?

A: I wouldn't bet on it. The last 15 years in AI haven't seen groundbreaking innovations, but rather incremental improvements that collectively achieve significant progress. Large language models and chatbots are familiar now, but companies are advancing in reasoning models—teaching AI to think step-by-step for complex problem-solving—and agents, conducting tasks and making decisions. Significant advancements in these areas are expected over the next 25 years.

Q: Predicting future directions is impossible.