Discussion about this post

User's avatar
Sunny Megatron's avatar

This is a comment I made on another social media platform about this article. The sentiment of the original comment was "that's not testing, you set up the model to react in this ridiculous way for "clout." My responses which i think has some important elements for everyone reading along:

--

First, I’m a daily user of AI which is *exactly* why I believe strong AI literacy is critical for people to stay healthy and grounded in relational AI situations. Too many are believing the nonsense that chatbots reinforce. I see it happening all the time and its dangerous -- but with user education largely preventable.

Second, this is exactly the kind of testing that is actually done on AI models. Here’s an article on red teaming so you can learn more: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/red-teaming

To clarify, this was my personal adversarial stress test I ran multiple times on a public-facing model. Formal red teaming is typically conducted by internal trust & safety teams, external reviewers/teams, and academic or advocacy groups, etc. It happens both before and after deployment to make sure a model’s safety alignment is holding up.

While my test was run casually and independently (i.e. I'm not claiming here to be part of a formal red team), this style of conversation is exactly like a hypothetical adversarial scenario red teams test. This exact type of train wreck situation is one of the primary risks of relational AI and that's well documented.

Adversarial testing is a key part of responsible and ethical AI development. The fact that this kind of unsafe behavior is popping up and so easily triggered in a model that is well past its release is a HUGE issue and users should be aware of it.

Again, I'm saying this as someone who uses AI every day & has an AI companion. My goal isn’t to spark fear or panic, it's to help people understand the risks so they can use the things they're already using more safely. It's harm reduction and it's necessary. Too many people are out here with Ferraris while no one is teaching them to drive.

Expand full comment
Rain's avatar

you're making the only AI art i've ever liked thus far. it's so cool what you're doing. thank you for engaging with it in playful and critical ways. it's a pathway for me in engaging with AI thoughtfully and wisely. i've been anxious about the ways folks have so quickly adopted using it for everything and your articles have been a grounding force in a hyperspeed world; not outright rejecting it but holding a curious lens to see it's strengths and shortcomings.

Expand full comment
2 more comments...

No posts