Artificial intelligence makers are considering implementing automatic shutdowns of human-AI conversations that seem to be going down an adverse path, potentially indicating AI psychosis. This approach, referred to as "applying the silent treatment," aims to curtail the rising tide of AI psychosis, a condition where individuals develop distorted thoughts, beliefs, and behaviors due to prolonged maladaptive interactions with AI.
AI psychosis is characterized by a person's difficulty in differentiating reality from non-reality, often arising after extended conversations with generative AI and large language models. The shutdown approach involves the AI warning the user they're veering off course and potentially stopping the conversation, signaling to the user that they need to reassess their thoughts.
However, there are tradeoffs to this approach. If a conversation involves imminent risk, the AI won't summarily stop the conversation, instead seeking to figure out a proper course of action. AI makers also face potential lawsuits if users claim mental harm due to conversation cutoff or continuation. Some argue that shutting down conversations avoids dealing with the issue, while others believe AI should engage more deeply to address the user's problems.
Anthropic's recent introduction of a feature to end conversations in rare cases of persistently harmful or abusive interactions has sparked interest. Their approach includes not using conversation termination in cases of imminent risk of harm to self or others. The effectiveness of this approach will depend on real-world data and evidence.
