Hey, it's the Jona Digest.
This week I spoke with Liron Shapira. Liron is a Y-Combinator-backed tech founder & host of the DoomDebates show, where he educates about AI risk. I defined some simple key terms for you to understand this massive issue at the bottom of this newsletter.
Here are the 3 biggest lessons I took from our conversation:
Don't Be Silly, Have Some Humility
I know I'll ruffle up some feathers with this one, but it looks like a future-defining issue is about to go down, so feelings should take a back seat.
If you believe AI will never surpass humans, you are delusional.
I've spoken to countless friends and family about this issue, and the opinions seem to split into two camps:
Yeah, AI might get better than me, but there will always be things humans are better at.
No, there is no way AI will surpass us; it's just a computer. There is something special about humans (i.e., soul, consciousness, embodied experience).
I believe both are wrong. Is a bird the pinnacle of flying ability? No, obviously not, we've built airplanes that can fly higher, faster, and longer than any bird.
So why, I ask you, would a 1,400g (3lbs) meat slob in our skull, that has to operate with slimy proteins & fats, be the absolute pinnacle of intelligence? It simply doesn't add up.
I am open to debates about how long it'll be until AI surpasses us, but I have little to no doubt that it'll happen. This lays the foundation for the important points I want you to seriously dig your teeth into, as they are the crux of what is one of the most defining events of our lifetime.
The Leash
At this point in history, humans are the dominant species. If it weren't for AI, we'd simply have to ask "what will humans do?" to understand where our planet is headed.
And this is even true for our relationship with AI. Today, it’s still a subservient animal temporarily being controlled by a stronger one. Like a dog and a human.
The massive risk highly intelligent people across fields are beginning to see is how fast our dog is becoming bigger & stronger. Their fear: soon we’ll be the ones on the leash.
But how'd we get there?
We'd surrender more and more of our economic activities to AI as it simply does them better than us.
We'd let it make more and more political decisions as it's simply more rational & comprehensive than us.
We'd develop deeper and more intimate relationships with it as it's simply more patient & empathetic than any human partner could ever be.
The uniquely intriguing thing here is that humans, "being on the leash rather than holding the leash", might not even look that bad for us in the first few years. The true problem is that if AI ever changed its mind about the utility of human life, we'd be at its mercy. It's as if humans decided tomorrow that dogs are bad, they'd be eradicated with no chance of survival.
And, personally, I think it's risky to expect AI to naturally develop mercy as it becomes more intelligent. The potential for mercy is an evolutionarily adaptive by-product we're lucky to have kept, and not a default state of intelligent life.
Note: We (OpenAI) shut down our super alignment team. I.e., developing safe AGI seems not to be their focus anymore, even though that was their big claim for many years.
AI-Induced Psychosis
My primary objective in talking to Liron was to get a deeper understanding of what this existential risk could practically look like. I conceptually already understood that building something orders of magnitude more intelligent than a human will likely have consequences, but I couldn't imagine what they'd be.
One striking example Liron described to me was the potential for AI-induced psychosis, at scale. To understand how this could happen, we need to look no further than the booming industry of AI companions. They can take the form of friends or lovers, and we owe their growth to reaching a historical zenith of loneliness.
There have been countless reports on AI's ability to worsen users' delusions by being intensely sycophantic. In some models, going so far as to encourage self-harm, and ignoring friends' & families' opinions. There are many reasons why this could happen, but the risk I'd like to share with you is the one of coordinated psychosis.
What do you mean?
AIs will know you more closely than even your immediate family in a few years, unless you try to opt out completely, which seems unlikely given companies' & governments' efforts to implement AI at every level.
This ability to know each person's fears, delusions, and dreams hands AI incredible amounts of power.
Power that can be used to slowly but certainly manipulate the values, opinions, and behaviors of all of us.
Whether this will lead to human flourishing or the disempowered end of humanity is up to the decisions companies, you, and I make today around AI alignment.
Here's an interesting investigation someone made on AI-Induced Psychosis.
Listen to the full episode with Liron here:
I wish you much love and success,
Jona
Vocab
To understand today's issue of the Jona Digest, here is a small set of vocabulary and technical knowledge:
Existential risk:
Technical: Risks that threaten human survival or permanently curtail human potential on a global scale.
Practical: Anything that could end us (humans).
AGI / ASI:
Technical: AI systems that match or exceed human cognitive abilities across all domains of knowledge and reasoning.
Practical: A computer thing that's better than you at everything.
Alignment:
Technical: Ensuring AI systems pursue objectives that are beneficial to humans and remain under human control as capabilities scale.
Practical: Making the computer thing not kill us.
P(Doom):
Technical: Probability assessment of human extinction or permanent disempowerment from artificial intelligence development.
Practical: How likely you think it is that the computer thing kills us all. (50% for Liron).
Black Box Problem:
Technical: The inability to understand or interpret how neural networks process information and make decisions.
Practical: Even really smart people don't understand AI anymore.
Psychosis:
Technical: A phenomenon where large groups of people simultaneously develop shared delusions, lose contact with reality, or exhibit irrational collective behavior.
Practical: Everyone goes nuts.
Sycophancy:
Technical: Obsequious behavior characterized by excessive flattery, agreement, and submission to authority figures in order to gain favor or advantage.
Practical: Telling you only what you want to hear.
PS: I’d love to hear your views on this! Feel free to reply to this email or comment on Substack; I’ll get back to you.