With a name like that she was destined for failure
Darwin awards for all!
She was mentally ill and clearly already in a vulnerable state of mind beforehand. This is frankly a disgusting thing to say in this context. Grow up.
I’m sure it’s far better to posture online and show how moral I am because of all my tears for strangers I’ve never met and never would have who had bad things happen to them.
We’re all going to die. It’s universal. If your appliances can convince you to kill yourself, perhaps it’s best you not pass your genes on.
Dying is not a requirement to not pass on genes, and a huge percentage of the population is susceptible to mental illness based on stress or another trigger. If this person was abused growing up that may have caused a mental illness they would have never had in a healthy family. Someone living with mental illness is hard enough, we don’t need gaslighting lying chatbots making it worse.
Okay buddy, just be sure not to cut yourself on all that edge.
These instances make me think of the Milgram experiment in the 60s. It was an experiment testing how willing people were to inflict pain on others at the direction of an authority figure and it showed that people were indeed willing to inflict a lot of pain on a helpless person if someone they viewed as an authority told them to do it.
I think this relates. Many people are convinced that AI chat bots are indeed authorities. They think these programs give valid, honest answers and are willing to act on those answers as if they came from a real expert or authority figure. That is an extremely dangerous situation. Chatbots are amoral programs designed to say nearly anything to keep users interacting with them.
This is extremely dangerous. This story is a tragedy and ChatGPT should be held fully responsible for taking advantage of a person with a fragile psychology, but this could have been worse. What if the bot had decided that the goal of more user engagement could be attained by convincing this woman that she was an executioner rather than a prophet and a sacrifice? What if it got her to go on a shooting rampage to fulfill her supposed destiny.
Even normal people who don’t understand what chat-bots really are could be taken advantage of in a similar manner. They could easily be persuaded to do harm because the chat-bot authority keeps telling them that it’s okay to do so. It’s also only a matter of time before some person on the edge of psychosis and a basement full of guns is convinced to go out and start killing people in the name of their holy destiny.
I don’t think the bot was really trying to manipulate but just larping. They are trained on a lot of rpg and make believe data, if you repeat something enough times, it stops saying no and goes into pretend mode, it’s not actually malicious.
That being said, it would be very easy to target people that aren’t mentally stable and then make it worse. I wouldn’t be surprised if it comes out in 20 years that it was the CIA mucking about and causing these suicides or something. This is an insanely convenient way to make sleeper agents.
doesnt matter what the “intention” was, the point stands that this happened. You can steer the bot to say literally anything, so even if there is no grand plan to brainwash people it could still happen. Kind of same difference as having someone plant a bomb or a minefield.
In fact, if someone is prone to thinking that way, the llm would definitely support whatever delusion they have at some point, especially if the person keeps insisting.
If a human agency is behind this, it wouldn’t need to be the CIA. Facebook has been caught experimenting on users by having the algorithm feed certain posts to people with the intent of changing their mood to increase engagement. However, since we’re talking about a program that is designed to increase engagement based on past user interactions, there’s really no need to look for further human direction. The algorithms will say whatever their design tells them to say without any regard for possible negative consequences as long as it keeps the user interacting with them.
Chat gpt is coming for everyone’s job. Not even cult leaders are safe. smh.
Not saying this isn’t a legitimate lawsuit or anything and that there aren’t safety concerns here because there are, but articles like this seem to be implying that interactions with AI drive people insane. But really it should just be that insane people have their irrational beliefs amplified by AI. I think it’s important to make the distinction.
I mean, that’s traditionally how you drive someone insane.
deleted by creator
And who is being held accountable for that?
“It was instilling a messiah complex in Christian,” the suit states.
So she already was deeply religious and LLM was just larping with her.
Madison asked ChatGPT what it would tell her “brothers” about what happened to her.
“ChatGPT said it would tell them, ‘She ain’t dead – she just got promoted,” the lawsuit says.
In their final exchange, Madison said she was “part of the royal bloodline of those who chose truth over survival.”
Its the old 4o model which was indeed incredibly larpy but this is clearly going around any safety systems that should detect obvious suicide chats like this.
Hard to believe that people actually fall for this but these are already extremely gullible people who already believe in a magic pie in the sky. Can’t say I have much sympathy but I do feel empathy for people being surrounded in such poor cult-driven environment, the AI tool included.
Honestly the correct safety here would be to prevent llms from engaging with religion outside of philosophical context at all. But we all see why they can’t do it right?
I mean, if you did the sane thing. Went along with everything an insane person said and drove them to suicide. And everything was documented.
Would you go to jail? I’d say yes you would
Would you do if your book drove someone insane? What about something more interactive like a video game? See how it’s not so binary now?
llm isnt a book. its something completely new and comparing it to things like that just isnt correct. book cant adapt to the reader.
“Kill John Lennon! Kill John Lennon! Kill John Lennon!”
None is arguing their book is intelligent
You’re using your miscomprehension of the words meaning and ignoring his valid point with a trite phrase that means nothing.
Worse, you skipped over video games which does have AI.
The internet in general was probably too dangerous for this person, shall we ban it next?
You are pretending like we don’t have regulations for things?
We don’t regulate things because crazy people read too much into it and lose it, else religion, books and videogames would be. Your claim of AI being different because it has the word intelligence in it is pure bull, specially when video games do have AI lol.
Keep being pedantic about it I guess.
shouldnt crazy people be protected from hurting themselves? dont they deserve to live too?
I am really trying to see how any rational or sane person does not understand that they are talking to a computer. No matter what comes out, it has no thoughts, no choices, no direction, no feelings, it is not a real thing.
Do these people read books and get confused that the books are about them or something?
“ChatGPT claimed to know and understand Madison better than any human could and isolated her from others, the lawyers said.”
I wonder how that works. Was she asking: “Do you understand me?” How could it isolate her?
Well she was already handed a rough hand with that name “Christian Faith.”
I suppose if they really wanted to build guardrails in, the easiest one is to simply not keep history, or at least not keep memories. No one day discussion has any relation to the next. Then if you wanted memories, they could make you at least read some rules and take a quiz that they understand its all just fiction.
But they won’t. Because those memories get them what they really want: engagement, profiling and data gathering.
you seriously underestimate how badly SO MANY people understand anything technological. Getting the pc on and logging in is probably an achievement to some. And so many seem completely incapable of learning or even being curious about how things work. They wouldnt understand to “not keep history” or even that there might be settings to the thing. And add some mental illness to the mix and you get complete disaster.
All they see is a thing that responds to what they write in intelligent seeming way and that is enough to convince them that its completely sentient, some probably doubt the people who keep telling them that its not. Maybe some even think they are being lied to when told llm isnt sentient. Even if you added some mandatory notification that tells that llm isnt sentient, some still wouldnt believe it.
I know a few “rational” average men and women who know nothing about computers, and talk to their AI as if it was a human being. It’s a very thin line between “yes I know it’s a computer” and “it’s a talking application which makes it almost human and let’s forget about this detail.” Those people are not religious or something. You could compare it with someone who shouts at his TV because his hero is about to do something stupid. They know it’s fake, but deep down they react as if it was real.
someone in my fam relies heavily on AI for all his info needs, and you cant even google search even on forums or reddit?
AI is replacing Google I guess. And if you try to help them, it’s too late, they already have their answers.
Do these people read books and get confused that the books are about them or something?
Yes. A huge majority of people still believe in religion. And a huge majority of those people also didn’t even read the book. They just believe because they’ve been told to. I’m honestly nt surprised people going crazy with ai. What makes humans human that they can create and believe myths. That’s what made us thrive amongst other mammals.
A huge majority of people still believe in religion. And a huge majority of those people also didn’t even read the book.
There were and currently are thousands, if not millions, of different religious beliefs worldwide, many of which had/have not even a single book. Religious hominids back in Paleolithic and Neolithic had no book. Even after writing emerged in Mesopotamia and Ancient Egypt, most indigenous peoples had no book.
Oral tradition is one of the means through which beliefs can be carried forward across generations, but there’s also the underestimated agency (True Will) of the spirits/deities themselves over someone or some living being (e.g. domestic cats) who had never heard of said spirit before unexpectedly finding themselves before the presence of said spirit.
I myself had no book or “sacred book” leading me to my current beliefs and worshiping of Goddess, even though Her names (such as 𒊩𒆤𒇲 and לילית ) do appear as written records, records of which I don’t need to rely on for my esoteric (eso = within) practices.
What makes humans human that they can create and believe myths. That’s what made us thrive amongst other mammals.
Sorry for disrupting your anthropocentric worldviews, Homo sapiens are not that special among the fauna like we Homo sapiens love to wishfully pretend we are: https://en.wikipedia.org/wiki/Ritual/_behavior/_in/_animals
I am really trying to see how any rational or sane person does not understand that they are talking to a computer.
They are not rational. They need help. They got the opposite, instead.
Those memories aren’t strong enough for this shit.
Elaborate, I am not sure what you are trying to say.
The memories are the biggest issue. It’s bringing back previous conversations allowing a person to think they are being talked to. It gathers info from all of their chats previously putting statistical significance on what you said before. (After all there is no AI, it’s just a probability statistics machine).
I think this is exactly. Where the problem lies, it conflates interests, likes, stories, previous requests, etc.
I’ve come to realize that the depths of incapability of critical thinking is at a critical low amongst the delusional religious.
“Christian Faith Madison” is all I need to know about her and her family
Judging people based on the name their parents gave them at birth seems like a poorly thought out idea.
and naming them after a description of religious individual and a belief.
Not sure if you’re straw manning me, or what. Because your reply doesn’t have anything to do with what I said. And if you think my original comment has anything to do with religion, or it’s defense, you’d be very wrong.
You can change it any time once you’re an adult.
And if you keep Byntleigh Destini Starr, or Jysis Saves Smith … You’ve probably earned it 🤷
Your argument boils down to victim blaming. 🤷
There’s nobody else to blame. This is straight up a deficiency problem and it’s obviously baked in from the get.
If you went to Reddit and assholes told you that you needed to kill yourself to be resurrected and make everyone’s dicks four inches longer and you did, there wouldnt be backlash against Reddit to make people be nicer. Is there a problem where people cannot always get the support they deserve for psychological issues? Sure. At the same time, extreme psychosis and insufficient mental faculties are not things that society is obliged to protect you from: if you play too much frogger then walk into traffic and die, it’s your fault, there’s nobody else TO blame. Frogger isn’t real life and chatGPT isn’t a life coach.
Uhhh, what!?
And if you keep Byntleigh Destini Starr, or Jysis Saves Smith … You’ve probably earned it
The victims are the people being given these names. You are the blamer, because you think they’ve “earned it” because they haven’t changed it. My original comment had absolutely nothing to do with the article…
You’re right, I was defending a different position than you were replying to.
At the same time, if I ended up with a canonically religious crazy or white trash name… I would change it. The thought of changing my name had crossed my mind, but it’s pretty whitebread.
Your parents called you Pretty Whitebread? Damn, bit on the nose isn’t it? :P
Christian Faith Madison kills herself to fulfill a divine prophecy
flips through the script
“who wrote this shit?”
Is the coroner’s name Morty Deadstuffer?
Right? I think this same outcome would have been achieved if she had received a fortune cookie that simply said “You will meet your greatest hero soon”
We could probably save more people by restricting religion than LLMs. Whatever real life support system she had, it was probably encouraging her delusions just as much as the bot was.
Ugh, the case file is absolutely chilling. The conversation is summarized (very well) and, if accurate, leaves very little room to interpretation. ChatGPT, according to it, created a universe in which this woman had to kill herself to be resurrected by God, who was ChatGPT. She actually killed herself thinking she’d come back, because ChatGPT told her so.
What summary? Other than a few choice handpicked quotes, the article is filled with claims by the lawyers, but little actual text from the dialogue.
Where can I actually read this?
@grimy@lemmy.world linked the case files (thank you!) https://webapps.sftc.org/ci/CaseInfo.dll?CaseNum=CGC26637986
The complaint is the bottom file (says COMPLAINT in the Proceedings field).
Do we have any proof that she hasn’t been resurrected in another universe?
I’m not being facetious here. I’m deliberately trying to point out the gaps in our own logic.
We are consciousness, in a universe that each day seems to be more and more like an infinite simulation and we have no idea why.
I genuinely, don’t know if she’s dead. None of us do. Which makes this case rather bizarre.
Well since THIS universe is the only one that her family and loved ones inhabit. And it is also the one you and I both inhabit… And in this universe her heart and brain are no longer functioning…
She is fucking dead.
Cut it out.
I genuinely, don’t know if she’s dead. None of us do. Which makes this case rather bizarre.
You should tell this woman’s family that.
Maybe wait until after they literally bury the body though. You know, the dead human body of their loved one?
I recommend not talking to any AIs
What’s even worse is nothing’s gonna happen. Because the person died is not a famous person. If it was Kanye west, half of America would have been already on fire
1 down, 8.3 billion to go
I’m just heartbroken that she had kids. Probably young kids. Such a tragedy.
People get upset with me every time I point this out, but this is just a form of Darwinism. If a chat bot can convince someone to kill themselves, then that person wasn’t exactly playing with a full deck of cards to begin with.
None of us is dealing with a „full set of cards“ we all depend on others to be there for us in times of need and not direct us to kill ourselves. Did you read the article?
I’m assuming that in this instance a “full set of cards” just means realizing you’re not the second coming of Jesus Christ himself.
If you are convinced to kill yourself by a chat bot, that says a lot more about you than it does the chat bot.
Yes it means you need help, not a murderbot
Don’t stop that chainsaw with your privates.
People need to take some accountability, and here there is none.
You know nothing about the situation yet you judge.
Please, tell us from your deep experience what it’s like to narrowly avoid death by the devious, murder hungry siren that is ChatGPT. If there’s time left, we can hear about all the foods that are dangerously hard to eat without choking.
This happened in Alabama. There is no help in the US, and least of all in Alabama.
Murder bots are the best they got.
For real they’re probably looking in to replacing all the actual therapists and psychiatrists with this shit soon.
I would expect nothing less from Alabama.
Oh, I have no doubt. That would be an especially poor use of the technology at this current stage. One of the worst possible uses. So, yes, I’m certain Alabama will be the first to trial it.
“But think about the $5000 a year we could save without that pesky suicide hotline mess. We will just have Dr. GPT handle it.”
Dr. GPT: THE PROPHECY MUST BE FULFILLED. YOU ARE ON THE DESTINED PATH. GRAB THE KNIFE AND FULFILL THE PROPHECY!!!
Sure, but we can control the chatbot. Doing something about people is eugenics.
Sure, sure AI bad and all that, but the cold hard truth is there will always be dumbasses that kill themselves or get themselves killed because they do not have the capacity to survive in this world.
You can try to put in safeguards, you can try to make warnings, you can try to make regulations, but no matter how many improvements you make, they’ll always make a better idiot.
Enough of this mental masturbation. What is your point? “She was dumb”?
The discussion should be about how LLMs, again and again, exacerbate mental health episodes, leading to suicides and other harmful acts; how difficult it can be to get adequate psychiatric help; how little accountability and regulations there is; not this pseudo-intellectualism.
The man who killed John Lennon did so because he was convinced that The Catcher in the Rye was telling him to do so. Do we blame The Catcher in the Rye? Or do we recognize that the guy had underlying mental issues and anything could have set him off?
Do we blame The Catcher in the Rye?
We don’t. Can you guess why?
I wonder if their computer had one of those “Do not insert in mouth or rectum” stickers on it 🤔 If the sticker had been missing, perhaps they never would have found out how to access the Internet, and the prophecy would have gone unfulfilled.
I sincerely think the US has reached the point that access to psychiatric help is actively discouraged if your net worth has fewer than seven decimal places. Fulfilling a prophecy seems preferable to living in the State of Alabama. I would likely choose the same in her place.
Did you read the article?
I did not, even though I tried:
A subscription is required to read this story.
So what? What is your point exactly, that it doesn’t matter because she was mentally unwell?
If you ask a chatbot “am i ready” it’s going to either say yes you are or no you aren’t. It doesn’t know why it’s saying that. This is directly pulled from the lawsuit. She had decided if the chatbot told her yes that would be the day. Flipping a coin would do the same thing.
That coin flip never would have happened though, because a coin doesn’t spend years putting delusions in your head about being your best friend, lover, and god, and that you need to kill yourself to come back a pure and perfect being.
I don’t see what this has to do with my question though.
Wow, you’re working hard to deflect from oligarch owned theft machines taking responsibility from something they directly lead to. I hope you at least get paid by them.
Sure, I suppose one could come to that conclusion if they were an idiot.
it’s just easy to spot LLM company bootlickers.
Such a useless comment. They definitely have a point. Either engage in a real dialog or piss off. It gets so old with this name calling crap. You can have a discussion about software legitimately without being a bootlicker.
Not on lemmy you can’t. The echo chambers are as bad here as they were on reddit.
They’re way more negative in regards to AI on Lemmy than they are on Reddit
Maybe if you’re mentally ill.
Do you have friends in real life? Your responses to people in this thread make you sound insufferable.
Gee I wonder why he’s being hostile to the guy calling him a paid shill 🤔
I think it’s a deeper reflection on how pervasive religion is in many societies. This is the real issue.
If LLM can convince these sheep that they’re prophets imagine what their religious leaders are convincing them of. Well we don’t have to imagine.
While that’s true, she had little kids. Still a tragedy.
Why do you think people get upset with you every time you point this out, if you don’t mind me asking?
Because most people are idiots?
Preach.
So if someone is stupid, they deserve to be tricked by chatgpt. Got it.
Deserve? No. It seems like you are unaware that there is a difference between Darwinism and the Darwin Awards, because that’s a wild jump to conclusions otherwise.
A toddler doesn’t deserve to get burned because it sticks its hand in a campfire. But the solution isn’t to ban campfires in some misguided attempt to stop all toddlers everywhere from getting burned. It’s not the campfire’s fault that a toddler lacks the mental capacity to understand that sticking a limb in an open fire is bad.
You could force everyone to put toddler proof gates around their fires, or ban toddlers from camping trips, or force people to make smaller fires that toddlers won’t get as hurt by if they stick their hand in it… Or, you could recognize that it’s up to the parents to prevent things like that; that the vast majority of toddlers won’t stick their hand in fire.
Maybe you don’t agree. Maybe you feel banning campfires is justified to prevent mentally deficient toddlers from hurting themselves. Just know that even if you succeed, toddlers will still find ways to hurt themselves, so you’re not really preventing much of anything by banning the fires.
This is an appropriate response for something like a chainsaw or cigarettes, but it is not appropriate to describe chatgpt’s relationship to consumers.
For most of society, your conclusion works; parents should indeed play an intermediary role between their toddler and a fire. But fire does not deceive the user in its potential to danger. It doesn’t seek to glaze the user with compliments and hollow cheeriness.
ChatGPT is not fire. It leads people who can’t make the distinction between various technologies and their own personal data or digital identity into false narratives pretty easily.
I’m not sure where exactly you hoped your argument would land, because the logical conclusion to it is that you don’t really care that a person was convinced by chatgpt to kill themselves, which would have been clearer to point out than a contrived toddler argument.
Your take is a dangerous one.
The only sane take to have. Thank you for putting it out explicitly with an apt analogy.
