Insurance companies are using AI. People will die.
So, life. Death. Taxes. Insurance companies and AI.
This is one very real, present way AI is going to kill people. Not by HAL-9000 or Borg malevolence, but with a heavy dose of group-think by insurance companies.
Life has brought me rather suddenly to the place that I’m interacting heavily with insurance companies about a medical issue for a loved one.
The customer web site has a chat bot, and of course the “cut-every-direct-costs-ignore-the-consequences” insurance company pushes you hard to use it. I did. The output was semi-coherent, non-resposnive and tended to just spit out canned responses that you would find on the customer service web site (because, to be sure, that’s what was trained on).
I can read the web site. When I call in, it’s because I’ve already done that and can’t get my question answered. I need a person who understands things, can explain what I’m missing, and help me resolve non-standard problems. But generally the people you get only know how to run a call script and maybe transfer you to other people who know how to run different call scripts.
I got one of those who I think was a real human being, but, going foward with text to speach/speech to text all tied to AI on the back end, I’m pretty sure it’s going to get to the point that you can ONLY interact with AI. Yay, Turning test.
So the person seemed to help me, seemed concerned, and promised to follow up on the issue I raised. We even got into an email exchange. 3 weeks later, no follow-up. No resolution.
I emailed back. The emails were all answered quickly, suspiciously quickly. It seemed to have no reference to the email exchange 3 weeks earlier, even though I had replied to the earlier message. It seemed unable to link ideas, reason or apply any common sense. I started getting the same snippits-from-web-site-docs replies. It told me that they both had and had not received something the doctor had sent them. Yup. AI again.
It also helpfully recommended that I set up a 3 way call with their billing people and the doctor involved to resolve the billing issue. So, the doctor is asked to sit on hold for the benefit of the insurance company, not spending time helping patients with medical issues and, not billing valuable hours while waiting on hold with the insurance company. Who’s running this show? For who’s benefit?
It would be interesting to see the results of some blind tests, A/B testing, etc. where customers got access to competent people to help them vs. only AI. I think we could measure the results in terms of mortality. That sort of testing would raise ethical issues (the value of a human life in terms of $).
Aside from raising my blood pressure (stroke anyone?), it’s going to delay needed care, give wrong/inconsistent answers (hallucinations etc), and cause caregivers, patients and medical professionals alike to waste time and figure out how to work around it.
But I’m pretty sure what’s going on is mid-to-upper level people inside insurance companies are just on the AI-hype bandwagon, driving it all the way down in their organizations, further dehumanizing customer service people before they lay them all off, and congratulating themselves on how forward looking they are and what good service they are providing.
People are going to die.
#58 of #100DaysToOffload take 3.1, https://100daystooffload.com/