# Who decides if a medical AI is safe?

> As AI moves from narrow imaging tools to chatbots and agents, who is responsible for proving it is safe to use on patients?

US regulators have widened exemptions for decision-support software while starting to design rules for generative AI, and Europe has chosen to layer the AI Act on top of medical-device law. Meanwhile consumer chatbots and NHS scribes are being governed by product terms, procurement rules and state laws rather than premarket review.

## Stops
1. **4 Dec 2024: FDA PCCP guidance** (Rule) The baseline: FDA lets cleared AI devices change within a pre-approved plan. [page](https://wheresthe.ai/rule/us-fda-pccp-ai-device-guidance/) · [source](https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial-intelligence)
2. **6 Jan 2026: FDA loosens oversight of AI decision-support software and health wearables** (Development) FDA widens the categories of AI decision-support and wearable software it will not review. [page](https://wheresthe.ai/d/us-fda-cds-wellness-guidance-2026-01/) · [source](https://www.fda.gov/media/109618/download)
3. **7 Jan 2026: OpenAI launches ChatGPT Health, linking medical records and wellness apps** (Development) A consumer chatbot takes in medical records while staying outside device regulation. [page](https://wheresthe.ai/d/openai-chatgpt-health-2026-01/) · [source](https://openai.com/index/introducing-chatgpt-health/)
4. **1 Aug 2025: Illinois bars AI from delivering therapy without a licensed professional** (Development) States step in for mental-health chatbots, some with outright bans. [page](https://wheresthe.ai/d/us-il-wopr-ai-therapy-ban-2025-08/) · [source](https://www.ilga.gov/Legislation/BillStatus?DocNum=1806&GAID=18&DocTypeID=HB&SessionID=114)
5. **2 Apr 2026: NHS England updates AI scribe guidance and launches supplier registry** (Development) The NHS uses procurement standards and a supplier registry to set a floor for AI scribes. [page](https://wheresthe.ai/d/gb-nhse-ambient-scribe-guidance-v2-2026-04/) · [source](https://www.england.nhs.uk/long-read/guidance-on-the-use-of-ai-enabled-ambient-scribing-products-in-health-and-care-settings)
6. **30 Apr 2026: Science study: OpenAI's o1 beat physicians at diagnosis on real ER cases** (Development) Evidence that LLMs can out-diagnose doctors on paper raises the stakes of getting oversight right. [page](https://wheresthe.ai/d/us-science-o1-ed-diagnosis-study-2026-04/) · [source](https://www.newswise.com/articles/landmark-test-of-clinical-reasoning-finds-ai-outperformed)
7. **7 May 2026: EU deal keeps AI medical devices under AI Act high-risk rules, delayed to 2028** (Development) Europe opts for dual MDR and AI Act scrutiny rather than a carve-out. [page](https://wheresthe.ai/d/eu-ai-omnibus-keeps-medical-devices-2026-05/) · [source](https://www.taylorwessing.com/en/international-life-sciences-newsletter/life-sciences-legal-lens-vol-2/update-ai-enabled-medical-devices-and-ivds-confirmed-as-high-risk)
8. **13 Jun 2024: EU AI Act** (Rule) The horizontal law whose high-risk duties now apply to AI medical devices from August 2028. [page](https://wheresthe.ai/rule/eu-ai-act-2024-1689/) · [source](https://eur-lex.europa.eu/eli/reg/2024/1689/oj)
9. **18 Aug 2026: FDA proposes a risk framework for generative AI medical devices** (Development) FDA sketches how it might review generative AI devices by autonomy and harm. [page](https://wheresthe.ai/d/us-fda-genai-device-discussion-paper-2026-08/) · [source](https://www.fda.gov/media/194242/download)

---
Canonical page: https://wheresthe.ai/line/who-decides-medical-ai-is-safe/ · A Line of Thought from wheresthe.ai