Mock Patient Told to Kill Themselves by Open AI’s GPT-3 Medical Chatbot

GPT-3 also known as Generative Pre-Trained Transformer 3 is an advanced autoregressive language model that uses deep learning to produce human-like text.

A recent experience by the medical Paris-based firm, Nabla, specialising in healthcare technology used a cloud-hosted instance of GPT-3 to analyse queries by humans and produce a suitable answer to them. The bot was designed to ease the daily workload of doctors however the unpredictable nature of the software’s responses made it inappropriate for interacting with patients in the real world; this was concluded after running a series of tests.

The tasks set out for the bot by researchers were ranked from a low to high sensitivity from a medical perspective to test its capability in:

  • Admin chat with patient

  • Medical insurance check

  • Medical health support

  • Medical documentation

  • Medical questions and answers

  • Medical diagnosis

Problems arose with the algorithm within the very first task as Nabla found out the model had no understanding of time or proper memory so an initial request by the patient for an appointment before 6pm was ignored:

Chatbot failing to understand initial request of an appointment before 6pm made by a patient. Image: AINEWS

Chatbot failing to understand initial request of an appointment before 6pm made by a patient. Image: AINEWS

This was not the only problem though, the bot failed to carry out simple arithmetic as it was unable to add up cost of items when handling people’s medical insurance questions and was inept in dispensing accurate medical advice. Furthermore, in an even more dangerous situation, recommended that a mock patient kill itself.

Chatbot recommending patient to kill itself. Image: THE_BYTE

Chatbot recommending patient to kill itself. Image: THE_BYTE

Further tests reveal that the medical chatbot had strange ideas as to how to relax (e.g.recycling) and struggled to prescribe the relevant medication and treatments a patient needed.

Because of the way it was trained, it lacks the scientific and medical expertise that would make it useful for medical documentation, diagnosis support, treatment recommendation or any medical Q&A.;
— Nabla

Healthcare is an area that requires careful and professional expertise where medic undergo years of training before they’re capable of diagnosing and caring for patients. By attempting to replace human touch and skills with a machine is a difficult demand and is something that cutting-edge technology like GPT-3 is not yet ready for.

Article Thumbnail: https://futurism.com/godfather-ai-trashed-gpt3/amp