1 in 6 AI users find it hard to stop chatting with AI, survey finds

The government has issued guidelines warning people not to become overly dependent on generative AI when using it for mental health counseling, and not to treat AI responses as medical diagnoses.

The Ministry of Health and Welfare announced Thursday that it had developed the "Guidelines for Using Generative AI for Mental Health Purposes" together with the Korean Neuropsychiatric Association.

Released to mark World Mental Health Day on Saturday, the guidelines aim to prevent side effects — including overdependence — and promote safe use of generative AI for mental health purposes.

The guidelines draw on a survey commissioned by the Korea Institute for Health and Social Affairs and conducted by Korea Research from Aug. 25 to Sept. 4. The survey covered 1,019 generative AI users between the ages of 15 and 79 who had used such services within the previous month.

The results showed that 81.3 percent of respondents first encountered generative AI for tasks such as assignments, work or information searches, while 68.8 percent also used it for emotional purposes, including casual conversation and mental health or psychological counseling.

Among teenagers aged 15 to 18, 69.2 percent said they used generative AI for emotional purposes.

Some 79.9 percent of respondents said they had received help or resolved a problem through AI responses or advice. Meanwhile, 16.3 percent said they had found it difficult to stop a conversation with an AI.

Between 50 and 60 percent of respondents took steps to protect their personal information when using generative AI — 61.0 percent filtered out private details before sharing, and 56.6 percent anonymized or omitted information that could identify specific individuals or organizations.

Some 38.0 percent said their conversation content could be leaked or used for unintended purposes. Meanwhile, 86.9 percent said the government had a role to play in regulating generative AI use.

People who perceived their own mental health as poor were more likely to treat AI responses as medical diagnoses. Only 11.9 percent said they verified or cross-checked mental health information provided by AI.

The guidelines incorporate these survey findings into a set of recommendations.

Before using generative AI for mental health purposes, users are advised to check the stated purpose, target audience and evidence base of the service, as well as how it handles personal data. For services that claim medical benefits, users should verify whether they have been reviewed or approved by a recognized authority.

Users are told not to enter personal information such as names, contact details or addresses, or sensitive documents such as medical records, prescriptions or test results, and to decide in advance how long and how far a conversation will go.

AI-generated mental health diagnoses or risk assessments should not be accepted as definitive medical judgments, and users are encouraged to consider other possibilities and perspectives.

Mental health information and advice from AI should be verified against materials from recognized sources such as academic societies or government agencies, or through consultation with professionals. Users are advised not to follow AI guidance that does not apply to their own situation.

Users are cautioned against overdependence on AI, and are encouraged to consciously limit their AI use if they find themselves relying on it more heavily or spending less time with family and friends.

For important decisions, users are urged to consult other people or professionals. In situations involving dangerous or inappropriate AI responses, or a mental health crisis, users are advised to seek help from people close to them or from professional services.

Separate guidelines were also prepared for children, teenagers and their guardians.

Children and teenagers are advised to recognize that they can stop a conversation with AI at any time, and to speak with someone close to them if they find themselves sharing worries or difficulties only with AI.

Guardians are encouraged not to ban children and teenagers from using AI outright, but to first understand why and how their child uses it, and then work together to establish safe usage habits.

Guardians are also advised to assess the appropriateness and validity of mental health information their child obtains through AI, and to monitor changes in the child's AI usage time and relationships with those around them.

The Ministry of Health and Welfare said it plans to develop education programs to help spread the guidelines across the field. Next year, it also intends to prepare tailored guidelines that account for the specific needs of high-risk groups, including people with severe mental illness.