Dwarkesh Podcast

Laatste aflevering:31 okt 2023 – Paul Christiano — Preventing an AI takeover

Dwarkesh Podcast

Deeply researched interviews www.dwarkesh.com

Door Dwarkesh Patel

Volg podcast

Luister via:

Afleveringen

Paul Christiano — Preventing an AI takeover

Paul Christiano is the world’s leading AI safety researcher. My full episode with him is out! We discuss: - Does he regret inventing RLHF, and is alignment necessarily dual-use? - Why he has relatively modest timelines (40% by 2040, 15% by 2030), - What do we want post-AGI world to look like (do we want to keep gods enslaved forever)? - Why he’s leading the push to get to labs develop responsible scaling policies, and what it would take to prevent an AI coup or bioweapon, - His current research into a new proof system, and how this could solve alignment by explaining model's behavior - and much more. Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes. Open Philanthropy Open Philanthropy is currently hiring for twenty-two different roles to reduce catastrophic risks from fast-moving advances in AI and biotechnology, including grantmaking, research, and operations. For more information and to apply, please see the application: https://www.openphilanthropy.org/research/new-roles-on-our-gcr-team/ The deadline to apply is November 9th; make sure to check out those roles before they close. Timestamps (00:00:00) - What do we want post-AGI world to look like? (00:24:25) - Timelines (00:45:28) - Evolution vs gradient descent (00:54:53) - Misalignment and takeover (01:17:23) - Is alignment dual-use? (01:31:38) - Responsible scaling policies (01:58:25) - Paul’s alignment research (02:35:01) - Will this revolutionize theoretical CS and math? (02:46:11) - How Paul invented RLHF (02:55:10) - Disagreements with Carl Shulman (03:01:53) - Long TSMC but not NVIDIA This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Shane Legg (DeepMind Founder) — 2028 AGI, superhuman alignment, new architectures

I had a lot of fun chatting with Shane Legg - Founder and Chief AGI Scientist, Google DeepMind! We discuss: * Why he expects AGI around 2028 * How to align superhuman models * What new architectures needed for AGI * Has Deepmind sped up capabilities or safety more? * Why multimodality will be next big landmark * and much more Watch full episode on YouTube, Apple Podcasts, Spotify, or any other podcast platform. Read full transcript here. Timestamps (0:00:00) - Measuring AGI (0:11:41) - Do we need new architectures? (0:16:26) - Is search needed for creativity? (0:19:19) - Superhuman alignment (0:29:58) - Impact of Deepmind on safety vs capabilities (0:34:03) - Timelines (0:41:24) - Multimodality This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Grant Sanderson (@3blue1brown) — Past, present, & future of mathematics

I had a lot of fun chatting with Grant Sanderson (who runs the excellent 3Blue1Brown YouTube channel) about: - Whether advanced math requires AGI - What careers should mathematically talented students pursue - Why Grant plans on doing a stint as a high school teacher - Tips for self teaching - Does Godel’s incompleteness theorem actually matter - Why are good explanations so hard to find? - And much more Watch on YouTube. Listen on Spotify, Apple Podcasts, or any other podcast platform. Full transcript here. Timestamps (0:00:00) - Does winning math competitions require AGI? (0:08:24) - Where to allocate mathematical talent? (0:17:34) - Grant’s miracle year (0:26:44) - Prehistoric humans and math (0:33:33) - Why is a lot of math so new? (0:44:44) - Future of education (0:56:28) - Math helped me realize I wasn’t that smart (0:59:25) - Does Godel’s incompleteness theorem matter? (1:05:12) - How Grant makes videos (1:10:13) - Grant’s math exposition competition (1:20:44) - Self teaching This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Sarah C. M. Paine — Why dictators keep making the same fatal mistake

I learned so much from Sarah Paine, Professor of History and Strategy at the Naval War College. We discuss: - how continental vs maritime powers think and how this explains Xi & Putin's decisions - how a war with China over Taiwan would shake out and whether it could go nuclear - why the British Empire fell apart, why China went communist, how Hitler and Japan could have coordinated to win WW2, and whether Japanese occupation was good for Korea, Taiwan and Manchuria - plus other lessons from WW2, Cold War, and Sino-Japanese War - how to study history properly, and why leaders keep making the same mistakes If you want to learn more, check out her books - they’re some of the best military history I’ve ever read. Watch on YouTube, listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript. Timestamps (0:00:00) - Grand strategy (0:11:59) - Death ground (0:23:19) - WW1 (0:39:23) - Writing history (0:50:25) - Japan in WW2 (0:59:58) - Ukraine (1:10:50) - Japan/Germany vs Iraq/Afghanistan occupation (1:21:25) - Chinese invasion of Taiwan (1:51:26) - Communists & Axis (2:08:34) - Continental vs maritime powers This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Dario Amodei (Anthropic CEO) — The hidden pattern behind every AI breakthrough

Here is my conversation with Dario Amodei, CEO of Anthropic. Dario is hilarious and has fascinating takes on what these models are doing, why they scale so well, and what it will take to align them. Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes. Timestamps (00:00:00) - Introduction (00:01:00) - Scaling (00:15:46) - Language (00:22:58) - Economic Usefulness (00:38:05) - Bioterrorism (00:43:35) - Cybersecurity (00:47:19) - Alignment & mechanistic interpretability (00:57:43) - Does alignment research require scale? (01:05:30) - Misuse vs misalignment (01:09:06) - What if AI goes well? (01:11:05) - China (01:15:11) - How to think about alignment (01:31:31) - Is modern security good enough? (01:36:09) - Inefficiencies in training (01:45:53) - Anthropic’s Long Term Benefit Trust (01:51:18) - Is Claude conscious? (01:56:14) - Keeping a low profile This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Andy Matuschak — The reason most learning tools fail

A few weeks ago, I sat beside Andy Matuschak to record how he reads a textbook. Even though my own job is to learn things, I was shocked with how much more intense, painstaking, and effective his learning process was. So I asked if we could record a conversation about how he learns and a bunch of other topics: * How he identifies and interrogates his confusion (much harder than it seems, and requires an extremely effortful and slow pace) * Why memorization is essential to understanding and decision-making * How come some people (like Tyler Cowen) can integrate so much information without an explicit note taking or spaced repetition system. * How LLMs and video games will change education * How independent researchers and writers can make money * The balance of freedom and discipline in education * Why we produce fewer von Neumann-like prodigies nowadays * How multi-trillion dollar companies like Apple (where he was previously responsible for bedrock iOS features) manage to coordinate millions of different considerations (from the cost of different components to the needs of users, etc) into new products designed by 10s of 1000s of people. Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes. To see Andy’s process in action, check out the video where we record him studying a quantum physics textbook, talking aloud about his thought process, and using his memory system prototype to internalize the material. You can check out his website and personal notes, and follow him on Twitter. Cometeer Visit cometeer.com/lunar for $20 off your first order on the best coffee of your life! If you want to sponsor an episode, contact me at dwarkesh.sanjay.patel@gmail.com. Timestamps (00:00:52) - Skillful reading (00:02:30) - Do people care about understanding? (00:06:52) - Structuring effective self-teaching (00:16:37) - Memory and forgetting (00:33:10) - Andy’s memory practice (00:40:07) - Intellectual stamina (00:44:27) - New media for learning (video, games, streaming) (00:58:51) - Schools are designed for the median student (01:05:12) - Is learning inherently miserable? (01:11:57) - How Andy would structure his kids’ education (01:30:00) - The usefulness of hypertext (01:41:22) - How computer tools enable iteration (01:50:44) - Monetizing public work (02:08:36) - Spaced repetition (02:10:16) - Andy’s personal website and notes (02:12:44) - Working at Apple (02:19:25) - Spaced repetition 2 This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Carl Shulman (Pt 2) — AI Takeover, bio & cyber attacks, detecting deception, & humanity's far future

The second half of my 7 hour conversation with Carl Shulman is out! My favorite part! And the one that had the biggest impact on my worldview. Here, Carl lays out how an AI takeover might happen: * AI can threaten mutually assured destruction from bioweapons, * use cyber attacks to take over physical infrastructure, * build mechanical armies, * spread seed AIs we can never exterminate, * offer tech and other advantages to collaborating countries, etc Plus we talk about a whole bunch of weird and interesting topics which Carl has thought about: * what is the far future best case scenario for humanity * what it would look like to have AI make thousands of years of intellectual progress in a month * how do we detect deception in superhuman models * does space warfare favor defense or offense * is a Malthusian state inevitable in the long run * why markets haven't priced in explosive economic growth * & much more Carl also explains how he developed such a rigorous, thoughtful, and interdisciplinary model of the biggest problems in the world. Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes. Catch part 1 here Timestamps (0:00:00) - Intro (0:00:47) - AI takeover via cyber or bio (0:32:27) - Can we coordinate against AI? (0:53:49) - Human vs AI colonizers (1:04:55) - Probability of AI takeover (1:21:56) - Can we detect deception? (1:47:25) - Using AI to solve coordination problems (1:56:01) - Partial alignment (2:11:41) - AI far future (2:23:04) - Markets & other evidence (2:33:26) - Day in the life of Carl Shulman (2:47:05) - Space warfare, Malthusian long run, & other rapid fire This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Carl Shulman (Pt 1) — Intelligence explosion, primate evolution, robot doublings, & alignment

In terms of the depth and range of topics, this episode is the best I’ve done. No part of my worldview is the same after talking with Carl Shulman. He's the most interesting intellectual you've never heard of. We ended up talking for 8 hours, so I'm splitting this episode into 2 parts. This part is about Carl’s model of an intelligence explosion, which integrates everything from: * how fast algorithmic progress & hardware improvements in AI are happening, * what primate evolution suggests about the scaling hypothesis, * how soon before AIs could do large parts of AI research themselves, and whether there would be faster and faster doublings of AI researchers, * how quickly robots produced from existing factories could take over the economy. We also discuss the odds of a takeover based on whether the AI is aligned before the intelligence explosion happens, and Carl explains why he’s more optimistic than Eliezer. The next part, which I’ll release next week, is about all the specific mechanisms of an AI takeover, plus a whole bunch of other galaxy brain stuff. Maybe 3 people in the world have thought as rigorously as Carl about so many interesting topics. This was a huge pleasure. Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes. Timestamps (00:00:00) - Intro (00:01:32) - Intelligence Explosion (00:18:03) - Can AIs do AI research? (00:39:00) - Primate evolution (01:03:30) - Forecasting AI progress (01:34:20) - After human-level AGI (02:08:39) - AI takeover scenarios This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Richard Rhodes — The making of the atomic bomb

It was a tremendous honor & pleasure to interview Richard Rhodes, Pulitzer Prize winning author of The Making of the Atomic Bomb We discuss - similarities between AI progress & Manhattan Project (developing a powerful, unprecedented, & potentially apocalyptic technology within an uncertain arms-race situation) - visiting starving former Soviet scientists during fall of Soviet Union - whether Oppenheimer was a spy, & consulting on the Nolan movie - living through WW2 as a child - odds of nuclear war in Ukraine, Taiwan, Pakistan, & North Korea - how the US pulled of such a massive secret wartime scientific & industrial project Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes. Timestamps (0:00:00) - Oppenheimer movie (0:06:22) - Was the bomb inevitable? (0:29:10) - Firebombing vs nuclear vs hydrogen bombs (0:49:44) - Stalin & the Soviet program (1:08:24) - Deterrence, disarmament, North Korea, Taiwan (1:33:12) - Oppenheimer as lab director (1:53:40) - AI progress vs Manhattan Project (1:59:50) - Living through WW2 (2:16:45) - Secrecy (2:26:34) - Wisdom & war This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com

Eliezer Yudkowsky — Why AI will kill us, aligning LLMs, nature of intelligence, SciFi, & rationality

For 4 hours, I tried to come up reasons for why AI might not kill us all, and Eliezer Yudkowsky explained why I was wrong. We also discuss his call to halt AI, why LLMs make alignment harder, what it would take to save humanity, his millions of words of sci-fi, and much more. If you want to get to the crux of the conversation, fast forward to 2:35:00 through 3:43:54. Here we go through and debate the main reasons I still think doom is unlikely. Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes. Timestamps (0:00:00) - TIME article (0:09:06) - Are humans aligned? (0:37:35) - Large language models (1:07:15) - Can AIs help with alignment? (1:30:17) - Society’s response to AI (1:44:42) - Predictions (or lack thereof) (1:56:55) - Being Eliezer (2:13:06) - Othogonality (2:35:00) - Could alignment be easier than we think? (3:02:15) - What will AIs want? (3:43:54) - Writing fiction & whether rationality helps you win This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com