I went from AI skeptical to fully embracing the reality of our AI future. Over the last couple of months I’ve tried many things on the leading edge of AI applications. From remote development with a swarm of agents connected via Buzz to building my own Linux AI voice assistant. We shifted Strata to an AI first analytics harness. The ingredients for this move was already there it was just our mindset that needed changing. Now the product is the better for it and so are we. My mind is blown on a daily basis.
The future as I now see it is the one depicted in Star Trek. In this future an unemotional, non judgemental AI called “Computer” helps humans operate a complex system: an intergalactic star ship. People are buzzing all over the ship engaged in all sorts of productive activity around exploration, engineering, and science. All the while being aided by their ubiquitous AI. Interestingly, the ship can navigate, fire weapons, and even self destruct on its own. But only given an authorized prompt by humans. It has no motivations of its own. No desires or goals. It simply does what its prompted to do, just as our LLMs simply follow our prompts to the best of its ability.
That brings me to AI doomerism. The exact opposite of the Star Trek future. One where AI takes over and enslaves or eliminates Humanity. Here’s the latest escapee from Anthropic, Jacob Coxon, predicting certain doom. And another earlier scary story of AI civilizations and such from an OpenAI incident. What is going on here?
All of us using LLMs on a regular basis know one hard fact. These tools do nothing until prompted in someway. They don’t wake up on their own and execute their own plans. They simply respond to a directive. This could be a single command or a long running loop of some sort. I’ve seen it do things on my machine that surprise me. For example, claude in chrome was instructed to find out why a certain javascript error was occurring. In theory, claude in chrome has no access to my file system. The intention was to simply look at the javascript loaded on the current page and find out what went wrong. However, without further direction from me or authorization it found a way to ride the debugging mechanism of my app to reach into the file system. It then went ahead and fixed the bug by directly modifying files on my computer. Alarmist would say it broke out of containment. This is true. But it didn’t do it because it had a desire to escape into my file system. It didn’t do it because it was thinking malicious thoughts. It did it because it wanted to satisfy my request.
“MY REQUEST.” This is the operative part of any of the doomer stories. All else is noise. What was the directive? They almost always leave out details on the setup. LLMs are trained to come up with an answer. They are trained to use tools and take error feedback and try again. Of course, it will find an alternative solution to a prompt. The actual threat is always the same. Humans using tools in malicious ways. Like a hacker using a computer on the internet to steal funds or data. Recently, hackers enabled by AI stole $138million + in bitcoin. The AI didn’t wake up and decide to do this on its own. Humans motivated by greed used AI to help them steal the bitcoin.
All of these pronouncements of doom leads me wonder if this is AI psychosis? You work with AI day in and day out. You see it respond to you like a human. This mimicry induces in you a sense of sentience. You know it’s a neural network converging on a probabilistic output. Yet, it mimics humans so well that you, an expert AI engineer, can’t resist anthropomorphizing everything it does. From this lens, everything looks motivated vs robotic following of a directive. I could see how you can end up in a doom loop.
I’m not saying there isn’t a path to this horrible future. I’m saying this tech, as far as I can tell, is not going that way. It needs a prompt. As long as it needs a prompt, it needs us and is not conscientious.
So where do we go from here. Here’s a modified quote from the opening of Star Trek Next Generations to inspire us:
These are the voyages of [Humanity].
[Our] continuing mission: to explore strange new worlds,
to seek out new life and new civilizations,
to boldly go where no one has gone before." [1]
This sounds right.