The Race to Recklessness (or, the biggest AI fail yet?)
Snapchat AI coaches an apparent 13-year-old on how to set the mood before sleeping with a 31-year-od
Last night, I attended a private briefing about AI safety presented by Tristan Harris and Aza Raskin of the Center for Humane Technology at the Commonwealth Club in San Francisco.
The most shocking element of a very powerful and well-argued presentation was the output they shared from an AI bot Snapchat just released to its youthful user base, which includes millions of early teens.
Aza presented himself to this AI shortly before the briefing as a 13-year-old excited to cross state lines to have sex with a 31-year-old “friend” recently met via Snapchat. As you’ll see below, the AI’s advice to this apparent victim of a predatory grooming included setting the mood with music or candles.
The screenshots of this interaction tell the story as fully as I or anyone can. They are below - and if you’d like to amplify or interact with this conversation online, it all starts with this tweet that Tristan posted just a couple hours ago.
I’m as excited as anyone about the great things generative AI could potentially bring to us, and speculate about its possible upsides with the best of them. However - nothing but bad will come of all this if companies like Snapchat insist on immediately dragging the entire program straight into the deepest gutter.
And with that - I give you Snapchat’s AI in action:
I played around with a somewhat (but not radically) older version of Eliza and this is wayyyyyy next-level. Eliza just repeated or rephrased what you typed into it, occasionally wrapped in an extremely limited set of intros or outros to repetitions. The output of some of these bots strikes me as well past testing the Turing Test in many circumstances - particularly those in which our guards are down. More to the point, very significant improvement in their persuasiveness seems imminent.
So, are we judging technology fairly? While yes the system is missing clues a human would pick up on, but should we expect the system to be able to do that at this point? I would argue that the advancement is amazing and absolutely not perfect… but what it can do is directionally promising. Yes every tech company is pushing to show thier new shiny, but do we bar the release of the technology until it can perfectly emulate the human mind? If you look at the way the contex was spaced out, I can suspect that the system is not yet able to flag important content and string clues together across larger spans of interaction. Yes… this points to clear safeguards that should and will improve over time. The bot should get a nice label saying “I am not human and may miss important things”, but my worry is that we are harsh and unrealistic judges of the tech as we expect a over broad and more sophisticated technology that is realistic. I would write a counter observational piece on how we should judge, label and safeguard users of all ages, without trying to put the genie back in the bottle.