AI in adult social care: new UK guidance, existing legal duties

What the new guidance expects

The guidance is deliberately practical. For AI-generated or AI-assisted care plans, it says human care staff should review outputs. For meeting and administrative tools, it similarly recommends human review. More generally, it asks organisations to define roles, accountability and oversight rather than allowing an AI output to become a decision by default.

Data governance runs through the guidance. Organisations are told to consider accuracy and quality, UK GDPR compliance, where processing takes place, what people are told and how confidential information is protected. The guidance warns against entering identifiable or confidential information into free online AI tools.

The ethical principles in the guidance include truth, transparency, equity, trust, accessibility, humanity and responsiveness. It recognises that biased systems can produce discriminatory results, and asks buyers to consult people receiving care and staff, define the objective, examine benefits and risks, and set measurable outcomes before selecting a tool.

It also says every organisation should consider an AI policy, including an organisation that decides to prohibit AI use. This is sensible governance advice rather than a newly created statutory duty. A policy will only be useful if staff understand it, approved tools are identifiable and the organisation can detect unapproved use.

Where binding duties may apply

For CQC-registered providers in England, the Health and Social Care Act 2008 (Regulated Activities) Regulations 2014 remain binding. Regulation 9 requires care and treatment to be appropriate, meet needs and reflect preferences. Regulation 12 requires providers to assess and mitigate risks to health and safety. Regulation 17 requires effective governance systems, risk controls and complete, accurate records. The new guidance does not amend those rules, but an unsafe or poorly governed AI deployment may affect whether a provider complies with them.

The application is fact-specific. A care-planning assistant may affect person-centred care and record quality; a falls-monitoring or acoustic system may affect safety and privacy; and a workforce or auditing tool may affect governance. Buying a product described as compliant does not discharge the provider's own regulatory responsibilities.

Data-protection law is a separate layer. Information about physical or mental health is special-category data under the UK GDPR. An organisation normally needs an Article 6 lawful basis and an Article 9 condition, together with any applicable Data Protection Act 2018 safeguard. It must also address fairness, transparency, purpose limitation, data minimisation, accuracy, retention, security, individual rights and any international transfer.

A data protection impact assessment is legally required where processing is likely to result in a high risk to individuals. Not every use of AI in social care automatically meets that threshold, but innovative technology involving systematic monitoring, vulnerable people, sensitive data or consequential assessments is likely to require careful screening and will often require a DPIA. The ICO recommends beginning the assessment early and running it alongside design and procurement.

Some products may also fall within medical-device law. Classification turns on intended purpose, claims and functionality, not simply on the use of AI or the supplier's label. Software intended for diagnosis, prevention, monitoring or treatment may require a separate assessment under the UK medical-device regime. A general administrative tool and a system producing a clinical risk or pain score should not be assumed to follow the same route.

Equality duties can also be relevant where AI affects services, employment or access. The guidance's warning about bias is not itself a legal finding. It should prompt testing for differential performance and disadvantage, reasonable adjustments and accessible alternatives under the legal rules applicable to the organisation and use case.

Practical actions for founders, suppliers and care providers

1. Define the intended purpose and limits in operational language. State the users, care setting, inputs, output, decision or action supported, foreseeable misuse, excluded uses and when a person must intervene. Reassess whether the product is a medical device whenever claims or functionality change.
2. Map the data and the parties. Identify each controller, joint controller and processor; the Article 6 basis and Article 9 condition; retention period; hosting location; international transfer; sub-processor; and contractual instruction. Do not accept a supplier's generic privacy statement as the full assessment.
3. Screen for a DPIA before pilot data is collected. Where one is required, complete it before high-risk processing begins, record the views of relevant people where appropriate, document residual risks and consult the ICO if the statutory prior-consultation test is met.
4. Make human review real. Name the reviewer, give them sufficient information and competence, allow them to disagree with the system, define escalation routes and record overrides. A nominal click-through approval does not address automation bias or weak evidence.
5. Validate performance for the people and environments in scope. Examine false positives and negatives, missing or poor-quality data, language and communication differences, disability, skin tone where imaging is used, cognitive impairment and changes in condition. Record limitations and the consequences of error.
6. Build an evidence-based procurement pack. Request the intended purpose, evaluation results, training and validation populations where relevant, data flows, sub-processors, security controls, incident history, downtime arrangements, model-update process, audit evidence and exit support. Contract for material-change notice, investigation cooperation and deletion or return of data.
7. Connect the deployment to CQC governance. Add it to risk registers, care-record controls, staff training, incident and safeguarding processes, complaints handling, business continuity and quality monitoring. Preserve evidence showing why the tool remains suitable for the care setting.
8. Adopt and enforce an AI policy. Identify approved and prohibited tools, permitted information, review requirements and a route for staff to report errors or unapproved use. Explicitly prohibit putting identifiable or confidential care information into unapproved public tools.
9. Monitor after launch. Track errors, overrides, complaints, near misses, unequal outcomes, security events, model drift and changes in supplier functionality. Set thresholds for restricting or withdrawing the system and review whether the promised outcome is actually being achieved.

The compliance conclusion

The guidance turns responsible-AI principles into a useful social-care checklist, but it does not authorise a product or decide compliance. The binding analysis still comes from the provider's regulated activity, the system's intended purpose, the data processed and the effects on people.

For suppliers, the commercial opportunity is strongest where product claims are narrow, evidence is reproducible and customers can integrate the tool into their own governance. For care providers, procurement is only the beginning: the organisation remains responsible for safe, person-centred use, lawful data processing and effective oversight.