Title: AI Will Annihilate Humans Within 10 Years Show: Joseph Carlson After Hours Guest: Joseph Carlson Date: 2026-SEP-10 URL: https://youtu.be/02z49hkJO-4 Length: 26:36 (1596 s) Note: Auto-captions from YouTube's "Show transcript" panel, grouped ~15s. Fillers (uh/um) and stutters removed; wording, numbers, names and every (mm:ss) cue otherwise verbatim (caption spellings such as "Jacob Coxin", "Oenheimer"=Oppenheimer, "enthropic"=Anthropic, "Dorcish"=Dwarkesh, "Yan Talon"=Jaan Tallinn, "dumerism"=doomerism left as captioned).
00:00 Today on the Joseph Carlson show, there's a good chance that AI will kill us all within 10 years. That's what some people believe, including Jacob Coxin, a former anthropic employee who just recently quit. And as he quit, he did a announcement of his concerns of AI and
00:16 his belief that AI is going to kill all of us. This was also in a highly coordinated effort with the media, creating a whistleblower moment that has gone hyper viral. Now, Jacob Coxin is on the news getting millions of views. He's all over the place, all over X. Many
00:31 politicians, including notable Democrats that are concerned with seizing control of AI, halting all progress and treating as though it's developing a nuclear bomb, include the likes of Bernie Sanders, who just included a bill that wants to make it so anybody developing AI would be subject to corporate death
00:47 penalty and persons shall be subject to more than 20 years in prison, citing that this is the same type of regulations that we have against unlawfully developing nuclear weapons. Many people are treating this as an Oenheimer moment that we're steadily marching towards creating a nuclear holocaust. That we're creating AI that's
01:04 getting ever stronger, ever smarter, ever devious, and in the process threatening humanity, raising the chances of human extinction over the next 10 years. And there's more and more people now sounding the alarms. Now, during this time of AI hysteria and fear-mongering and outright panic
01:21 calling upon the end of human civilization within 10 years, I want to offer a different approach because what I see here is highly manufactured and contrived. And to understand how this came about, how all this viral concern about AI happened, I think it's good to go through the timeline. So, let's first
01:38 back up a little bit, just a day ago, and let's look at the original post. This is September 8th, and it's from a Jacob Coxin. Now, I've never heard of Jacob Coxin, and neither have you because up until now, he wasn't notable by any means whatsoever. In fact, even when we look at his ex account, we go into his history. This is the only post.
01:58 It's the only one. And he didn't even have an ex account until just a couple months ago. For now, we'll go through the post itself. Now, before reading it, I just want to note the viralness of this post. It has 152 million views. Even if these are just impressions, that's a lot. 150 million is one of the most viral posts I've ever seen. It has
02:16 17,000 comments, 187,000 reposts, 741,000 likes. So, by any measurement, this is one of the most viral pieces of content currently floating around the web, and now it's making itself into the mainstream. He says here that I resigned from Anthropic today. I spent the last 3
02:33 years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improvement, super intelligence, and gambling with our lives. more thoughts below. Now, right off the bat, I noticed something here. This is a really well-written post
02:51 in terms of trying to go viral. This is how you do it. You have a nice bait, a nice piece of bait, something that people can grab on to. They're racing straight to self-improvement super intelligence and gambling with our lives after noting that he worked for both Anthropic and
03:08 Open AI. So he's named dropping huge names and then he's saying that they're gambling with our lives and then he has a lot of other terms like super intelligence in there. He continues on saying do not underestimate the power of this technology. These will soon be superhuman systems that can hack
03:23 anything, revolutionize any field overnight and acquire real power and resources. We have all witnessed the progress in each of these domains and progress is not slowing. The people building AI earnestly believe that it could kill all of us by the end of the decade. This is not a marketing stunt.
03:40 If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible. But I hear the same people express fear privately. No other human activity possesses this level of danger. A common response is quote, "If they truly believe this, why are they still building it?" At OpenAI,
03:58 many have not deeply internalized the civilization stakes. At Anthropic, the stakes are well understood, but they are locked into the race to get there first. They believe no one else will act responsibly. So they must do it themselves despite the risk.
04:14 So right off the bat, you have a group of people working at Anthropic that believe they are doing the most important thing in the world. They are the most important people in the world. They've elevated themselves that way. They see themselves as in this race. And what is the race for? It's to
04:30 save all of humanity. That's what they're doing. the enthropic employees. They're not just employees at a company creating LLMs and great technology, iterating on it. No, they are literally saving humanity. Did you know that? Cuz they're telling you
04:45 that's what they're doing at the company. They know that civilization is going to be destroyed. It's only a matter of time. There is going to be super intelligence. And if they don't get there first, someone else will. and that other person is not as trustworthy as them because they're the only ones
05:01 trustworthy and responsible enough to have it. So, you can see how they view themselves. Anthropic employees clearly view themselves as the most important people in the world saving humanity in this giant race. They probably watch the Marvel Avengers movie and say, "Look, I'm like Captain America. I'm like Iron
05:18 Man. I'm saving the world from evil Open AI or evil X or the open- source LLM." And you may think that this is an exaggeration, but I can cite more examples. Here's another anthropic employee. This is Evan Hubbinger that
05:33 says, "Jacob is correct here. We really do earnestly believe AI could kill all humans. I personally think it is above a 10% chance within the next decade. I believe anthropic is trying its best, but we do not yet have a plan to solve alignment for super intelligence and are
05:49 not clearly on track." And why is he working at Anthropic even though he thinks the technology could kill us all? because he believes if he doesn't get there first, then he has no chance of saving humanity against the other evil LLMs. And this is Evan Hubbinger. He's
06:04 the one that's saying we have above a 10% chance of human annihilation within 10 years because of AI. But he's the one that's going to save us from this outcome. Now, not to get too sidetracked because Jacob Coxing continues on in this viral post. He says, "Acepting this race and entering the endgame is a
06:20 hubistic gamble that should not be launched from a private company Slack. attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available. I am optimistic about the potential for coordination. Warning shots like the hugging face attack made pacing agreements between the US labs more viable. I don't feel like we're on
06:40 track to prevent a global race which may require costly action such as a temporary ban on improving model capabilities. See, suddenly Jacob's viewpoint here seems to align far more with Democratic senators like Bernie Sanders. A temporary ban on improving
06:56 model capabilities is a ban on AI. So now he's going from quitting anthropic to suggesting policy that is the exact policy of Bernie Sanders which was released 3 days prior. In fact, immediately after this post went viral, as politicians often do, they jumped on
07:12 it. And so far, the count is at least right now 22 politicians have directly called for AI regulations and legislation based on this single post by Jacob Coxin. You have the entire group here. there's a couple Republicans,
07:28 mostly Democrats. Most of them are calling for a halt in AI progress and much stricter AI regulation. Jacob is also taken to the media doing a complete blitz going on to all the morning shows and evening shows of the day. He started things off with Anderson Cooper and notice the things that Anderson Cooper
07:44 focuses on the most. The things that he know will play well to his viewers. 27year-old outlined his concerns in a series of social media posts when he quit, writing in part, "The people building AI earnestly believe that it could kill us all by the end of the decade." That's the headline news which Anderson Cooper repeats multiple times throughout this interview. It's the one
08:04 line that he just hangs on over and over again. first of all, I assume you are saying literally that you believe that AI could kill us all by the end of the decade. How would that happen? Like what would that actually look like?
08:24 Yeah. So, the precise scenario sounds a little bit like science fiction. it sounds like something that's not real, but I think it is frighteningly real. And it's easiest to understand it if you think about real things that happened two months ago. So two months ago, open AAI agents AIS hacked
08:40 into third party infrastructure entirely of their own accord and this was like a concentrated hacking spree that they carried out of their own valition. Now he grossly misrepresents what actually happened and so did the retelling by Dorcish that used intentionally inflammatory language and provocative
08:56 language. What happened with the hugging face hack wasn't nearly as interesting as people might think. The system was told to hack a system and then the agents worked on hacking it. The monitoring systems could have easily stopped it and then the letters that the agents left at the end were logs by the
09:15 bots. This is not some great civilization being created. There was no civilization. There were agents doing something that they were instructed. So this was not entirely on their own valition. The agents did not get together and vote that they wanted to
09:31 hack a random system. So the people using this example, the hugging face example to promote AI dumerism and AI fear are using a bit of truth that an AI system was told to hack something and it did it to say now that they're creating their own civilization. They're doing it
09:47 on their own valition. AI is just spiraling out of control. it's already super intelligent and it's going to take over the world. That's not what that story portrays. But that isn't the only point that Jacob makes. He continues on. And I think that if you extrapolate into the future the level of capabilities of these AIs with the same independent valition, they could cause extreme
10:07 havoc. For example, hacking critical infrastructure, building extinction level bioweapons. There's a lot of ways that the AI could actuate itself in the world. Now, this is another area where Jacob greatly exaggerates. He says that AI will make
10:23 it much easier for critical infrastructure to be hacked and to create biological weapons. And the logic behind this, I'll grant him is pretty simple. If you understand biology better, which AI does, it helps us understand biology. Understanding a knowledge means that we can use it in
10:39 both ways. We can use it for good and we could use it for devious or adversarial ways or our adversaries could create biological weapons. So the logic seems pretty straightforward until you get into some of the details. For example, one of the problems with this thought that AI is going to suddenly make it so that everybody can create a biological weapon in their dorm
11:01 room. It just doesn't hold up. And the reason why is because what if I was to tell you that right now currently there is information online of how to create biological weapons, really dangerous ones, ones that would kill lots of people. So why aren't bad actors
11:17 creating them? There's already the knowledge to do that today. We don't need to wait until AI makes it more available or makes it so that you have it laid out in chatbt. It already exists. People can find it. The problem is that creating a biological weapon that could wipe out human civilization
11:33 is much more difficult than said. You need a list of dependencies needed for this project that are incredibly difficult to get, including all the biological material. Where are you going to get that? Well, good luck with it. You need laboratory competence. You need to be able to replicate tests. You need to have the experiments working repeatedly and reproducibility. You need
11:52 to be able to control for outcomes that you didn't expect. You also need to be able to deal with containment and keeping the organisms themselves stable and to be able to have an appropriate way to transmit it. That's a tall task. It requires a lot more than just having a couple ingredients or an instruction set on a screen. And this is part of the reason that biological attacks aren't more common is because it's incredibly
12:12 difficult for anyone to actually pull them off. Making AI have our understanding of biology does have a potential downside. I'll grant you to increase some level of theoretical risk of biological attack. But that's also without mention of the immense upside.
12:27 If we understand biology better, we can also theoretically treat every single problem, every single issue that we have with a better understanding. Literally, every single illness that we have, every single therapy, every disease, everything that we have, we'll have a
12:43 better understanding of treating. The upside from AI's advancement is also virtually unlimited. And it's also far more practical because everybody can study that in the open. they don't need to do so in secrecy with a secret lab, a layer that nobody else is looking at.
12:59 This is another example where the downside is being grossly overexaggerated with no mention of the counterpart or the upside. No mention that this technology could help millions of people with new understanding and new therapies. And I know this might be a little bit nitpicky, but I think it's
13:14 important to highlight because he's not just saying AI could raise the danger of biological weapons. That's not what Jacob's saying here. And I think the verbiage, the actual words that they're using to cause this type of fear and panic is important to highlight. He says extinction level biological weapon structure building extinction level bioweapons.
13:37 Extinction level. That's what AI is going to do. It's going to make the human race extinct, which is just ridiculous. Even looking at biology and looking at different diseases or biological weapons, they do not impact everyone the same. We have different biology. Some people are immune to different diseases. They're immune to
13:53 different things. They react differently. That's why different diseases don't affect everyone the same. And some people they don't affect at all. Also, there's geographical isolation. There's people in different locations. There's how people would react if there was a biological outbreak. So, he's stating things as dramatic and as dangerous and as scary as possible, even when it makes no realistic sense. And this is the exact
14:14 recipe for dumerism, fear-mongering, and intentional effort to stop something or to cause regulatory hurdles. Not only does he mention extinction of the human race by developing biological weapons, but he also mentions that AI will allow bad actors to take over critical
14:30 infrastructure. He mentions critical infrastructure. Now, there is a lot of problems with this as well. Just a few things that come off the top of my head, which is one of them. critical infrastructure is already under attack. It's already under attack by
14:46 state actors. Does Jacob think that actors in Russia and hackers aren't already looking at critical infrastructure in the United States? They're not already looking at exploits. Microsoft, Apple, all these companies, Amazon, they're being hacked every single day. hacking attempts every
15:03 single day from statef funded actors the biggest institutions that you can think of people with unlimited resources sure AI prompt can help them it can make them maybe a little bit more effective but you also have these companies having AI as well do you not
15:19 believe that they're going to increase their security as a result and then maybe you can say well Microsoft and Amazon and Google and Apple those aren't critical infrastructure what about the power grid what about the telephones what about the internet. Well, that's even more difficult to hack because none of that is unified. It is heavily geographically segregated. It's
15:40 in isolated parts. A state actor or an adversary with an AI would have to hack each individual part of it. It's part of the reason that the infrastructure hasn't been hacked more often. It's just incredibly difficult. You're not hacking one big power plant that supplies power
15:56 to the entire United States. That's just not how it works. So at most you might be able to take down power in one city for one period of time until they cut off the bad actor and get the power back running. This is the exact reason why state actors in different countries aren't able to hack the US more
16:13 effectively. And even in other territories like with Ukraine hacking Russia that took 6 months of time that took getting passwords, fishing attempts, taking over a large-scale attempt. These are not easy things to
16:28 hack critical infrastructure. But if you're just looking at these interviews, if you're just looking at what Jake is saying, it comes across as though it's just going to be easy. Oh, AI will do it. You'll just be able to plug it in, hit a couple buttons, and then boom, you control all the infrastructure in the United States. Just that easy. Or you
16:44 can take over massive websites at Facebook and Google and Amazon, and they have no hope of getting it back. Now, this media tour continues on. Jacob is going on more and more appearances even today. Each one of them racking up millions of views as he fearmongers more and exaggerates the downside and
17:00 inevitable demise of human civilization unless regulators get involved. And this is where we get to potential motivations and the cause of all of this. What will happen because of this viral moment? Well, there's a couple different ways this could go. One of them is that we look at the motivations or the incentives rather of the different parties involved here. Now, with Jacob
17:19 specifically, I don't know his motivation. Maybe he is sincere. He could be sincere that he's concerned about AI. There's many people that are frightened of technology in general and they have been in the past as well. But when we look at other parties involved, the incentives become far more
17:35 clear. Anthropic has been on an outright campaign for regulatory capture for a long period of time. In fact, one of the people that funded Anthropic was Yan Talon, a notable skeptic of AI, someone that doesn't even really like AI. And the reason that he originally funded or
17:50 he originally invested specifically to have control over it. In fact, at the same time that he was funding it, he was a big part of survival and flourishing fund which is the single biggest promoter and funer of the regulatory effort of AI. So there's already people since the very beginning that were
18:06 frightened about AI that specifically invested in AI to have control over it, knowing that you can't really control something unless you have some stake in it. You have some equity in it. But this isn't where it ends. We also have anthropic making it clear that they want these regulations and a lot of what this
18:22 amounts to looks like a campaign for regulatory capture. Regulatory capture is when a company works in coordination with the government to create burdensome regulations that create a higher barrier to entry. This makes it so that the incumbents, the companies that are already large and profitable and have
18:37 huge market share, they have an even bigger advantage knowing that smaller companies have a harder task to deal with with all these burdersome regulations to be able to catch up. So, it creates a strong incumbent advantage. And this seems a lot like what Anthropic is doing today. They're continually advocating for strict regulations, complex ones that would be very
18:57 difficult for any other type of competitor to compete against. And they've also been very reluctant to have any type of open-source. So, this is a very insular strategy. Now, there's also a benefit here for politicians. For example, we look at what Bernie Sanders has been doing for some time. For Bernie
19:14 Sanders, the goal here is to look like you're standing up against big companies. That is the Democrat framing. You have the billionaires, the big bad evil rich people working at the big bad companies that are creating the evil technology. The evil technology is going to replace your job. It's going to make you lose your job. And it's also going to create bioweapons and mass destruction. It's going to destroy critical infrastructure. And you're here
19:37 to stop it. Isn't that an incredible framing? You're the hero standing between the little guy and the big bad evil company. It's poetic political framing. And Bernie Sanders, like many other politicians, are cornering this market. They're fulfilling this demand.
19:53 He is pitting himself as stopping the AI Oenheimer moment. And if you think that framing is incorrect, it's not. It is incredibly accurate. Bernie's new bill, just included a couple days ago, literally includes language similar to Oppenheimer. In Bernie's bill, he also misinterprets and misrepresents the
20:11 hugging face hack by OpenAI, grossly exaggerating it and using many of the same talking points that a YouTuber made. And then he also concludes here that quote, "Shockingly, it took OpenAI nearly 2 weeks to discover this breach. AI has also recently been used to create new viruses, demonstrating AI could be used to develop new bioweapons that might result in the deaths of tens of
20:32 millions of people." So you can see the exact same talking points, the same stories, the same narrative is being sold wholesale to these politicians. This goes on with this outright ban which would completely halt all AI development within the United States for any person or entity that attempts to
20:48 violate or circumvent the pauses and prohibitions laid out in this bill. Entities shall be subject to corporate death penalty and persons shall be subject to no more than 20 years in prison which is similar to existing penalties related to the unlawful developing of nuclear weapons. Bernie Sanders is here to stop us from the Oppenheimer story. Now you may just look
21:10 at this and say well that's just Bernie Sanders. That's Bernie being Bernie. You know he is a little bit provocative. And that's the same with basically every politician at this point. It's not a right or left thing. We have Trump. He's provocative as well. Bernie Sanders is provocative. And maybe you can just write this off and say, "Yeah, that's just Bernie being Bernie." But I don't think that's the case. I think
21:30 that Bernie Sanders may be on the more extreme side of this. But what you can see is that this movement against AI is becoming far more mainstream in the United States. I believe it's a big problem. I think it's the wrong direction entirely. This is an incredible technology. And while it does
21:47 have some limited downsides that are being grossly exaggerated here, the upside of this technology is extensive. It is virtually unlimited. The amount of good things that we can accomplish with this technology if we don't shoot ourselves in the foot would be
22:02 incredible. And unfortunately, most of the remedies proposed here for having regulatory capture where only a couple companies control it would actually make it more dangerous, not more safe. technology that is widely distributed has more safeguards than ones that are controlled by a couple people that believe they're the ones only good
22:19 enough to control it. We also have the fact that even Bernie Sanders suggesting that the US should stop development of AI doesn't stop other countries from developing it. China's not going to stop. China views AI positively. They like the technology. They haven't had
22:34 the fear-mongering and dumerism. They see the benefits of it that it's increasing their productivity and their efficiency. It's going to make them more wealthy. It's going to increase their knowledge and understanding of different subjects. AI has incredible benefits that other countries are embracing at the very same time that we're turning
22:51 against. And unfortunately, this type of thing works. It's effective. These fear-mongering campaigns are changing public opinion rapidly. They are effective. And you're seeing the internet do the same thing. Even in my own channel, investors in these
23:07 companies, the ones investing in Google, in Amazon, in Meta, in Microsoft, the ones that are building data centers, I had one of the biggest reactions when I was pro data center. If I look back one month ago, I had people leaving me comments, dozens of them like this with
23:24 hundreds of upvotes, when I said that I was pro data center, here's how they reacted. I would love to hear your opinion when some company decides to build a data center behind your home. This is like the gotcha that many people are posting. Another comment, again, there's dozens of them, said, "Build data centers in your backyard if you
23:41 love them so much, Joseph." And this has 101 thumbs up. This was me suggesting that data centers are a good thing because AI is a good thing because it's good for the economy. It's good for knowledge. It's going to be highly efficient. It'll create more jobs than it destroys. There's a tremendous amount
23:56 of benefits that I can outline here. And the reaction online is, well, if you love data centers so much, build them in your backyard. This is the media. This is the type of things that it's selling people. And this argument doesn't even make sense. I'm also pro electrical substations because I like having
24:13 electricity. I'm also pro chemical labs and heavy chemical labs because I like the things that chemicals make. They make a lot of products that we use on a daily basis. It's important to have chemicals. I'm also pro prisons because I like having rapists and murderers
24:29 locked up in a confined place and not free in society. Do I want a prison in my backyard? Do I want a chemical lab in my backyard? Do I want an electrical substation in my backyard? Of course not. I don't want it in anybody else's backyard either. And this is precisely why cities have something called zoning.
24:47 City planning and zoning is important. The industrial zoning is where they have industrial projects like electrical stations like heavy chemical labs like distribution centers and commercial projects and construction. That's where they put that stuff because people don't want to live next to it. Then you have residential zoning where they put parks.
25:06 Obviously, nobody wants to have their backyard against any industrial zoning, including data centers, including a prison. I also wouldn't want that in my backyard. I wouldn't want a chemical lab. In fact, I'd pick a data center before I'd pick a chemical lab in my backyard and so on. That's why they zone
25:22 those things in a different area. See, these type of arguments that go online that are promoted by social media are picked up by so many people. They're so effective when they don't even make sense on the surface. These are intentionally made to create false social proofing to be easy and viral to
25:38 spread. And now that we already have this antagonistic view against AI and data centers, this one that is going to slow down the United States, we have an opening for other countries to gain the lead. And China knows this. It's why they've latched on to it. They have an army of bots that are trying to sew more
25:54 doubt and just act as a catalyst for this type of negativity. And we have people lining up to follow. Now, fortunately, up until now, the biggest doomers and fearmongers have been largely ignored as progress has continued. But it looks like we may be getting to a breaking point. It looks like the tides could be turning for real
26:10 against AI. And hopefully if that happens, hopefully we can continue to see this technology advance. It's important to the United States. It's important to the world. This technology does have immense potential to do good. And I don't see any of that being highlighted by any of the commentary from these people. Only doomerism, only fairing, and only resentment towards the
26:31 technology itself. And I think that's going to be a problem. That's it for this time. See you in the next one.