Lines of Thought

Who decides if a medical AI is safe?

As AI moves from narrow imaging tools to chatbots and agents, who is responsible for proving it is safe to use on patients?

US regulators have widened exemptions for decision-support software while starting to design rules for generative AI, and Europe has chosen to layer the AI Act on top of medical-device law. Meanwhile consumer chatbots and NHS scribes are being governed by product terms, procurement rules and state laws rather than premarket review.

  1. Rule · In forceUS
    FDA PCCP guidance

    The baseline: FDA lets cleared AI devices change within a pre-approved plan.

  2. Rule changeHealthUS
    FDA loosens oversight of AI decision-support software and health wearables

    FDA widens the categories of AI decision-support and wearable software it will not review.

  3. LaunchHealthUS
    OpenAI launches ChatGPT Health, linking medical records and wellness apps

    A consumer chatbot takes in medical records while staying outside device regulation.

  4. Rule changeHealthUS-IL US
    Illinois bars AI from delivering therapy without a licensed professional

    States step in for mental-health chatbots, some with outright bans.

  5. PolicyHealthGB
    NHS England updates AI scribe guidance and launches supplier registry

    The NHS uses procurement standards and a supplier registry to set a floor for AI scribes.

  6. ResearchHealthUS
    Science study: OpenAI's o1 beat physicians at diagnosis on real ER cases

    Evidence that LLMs can out-diagnose doctors on paper raises the stakes of getting oversight right.

  7. Rule changeHealthEU
    EU deal keeps AI medical devices under AI Act high-risk rules, delayed to 2028

    Europe opts for dual MDR and AI Act scrutiny rather than a carve-out.

  8. Rule · Partly in forceEU
    EU AI Act

    The horizontal law whose high-risk duties now apply to AI medical devices from August 2028.

    Also on Project Meridian and the race to govern military AI, Transparency, not licensing

  9. PolicyHealthUS
    FDA proposes a risk framework for generative AI medical devices

    FDA sketches how it might review generative AI devices by autonomy and harm.

Where this line could go next

Connected developments not on this line