Health information can leak through obvious uploaded records, retrieval metadata, text embeddings, diagnostics and copies sent to subcontractors. Limiting the visible answer is not a complete privacy plan.
Document the entire route
List capture, extraction, redaction, hosting, inference, output moderation, storage, human review, audit logging and deletion. Identify who has access at each stage. Determine local health information and general privacy duties separately; a product being labeled medical-grade is not proof of compliance.
Minimize the payload
Prefer the smallest necessary section of a record, use coded identifiers where practical and avoid supplying full patient histories for narrow extraction tasks. De-identification may reduce exposure but can fail for rare combinations or free text. Test whether embeddings and searchable indexes retain identifiable information.
Keep clinical and privacy assurance separate
FDA clinical decision-support guidance addresses whether certain functions are medical devices under US law. It does not certify that a provider’s data retention or cross-border sharing complies with privacy law. Likewise accuracy testing does not establish a lawful data-processing purpose.
Control access and onward transfers
Enforce user-specific permissions for retrieved records, vendor contractual review, limited support access and incident response. Ask exactly what is retained by the inference provider and subprocessors. If answers are unavailable, do not send clinical records based on a general security badge.
Example
A clinic pilots discharge-note drafting using de-identified test cases. It maps all subprocessors, confirms permitted purposes and the applicable patient-data requirements, then restricts production use to approved users with clinician review. The fact that test notes were safe does not authorize unrestricted real-record input.
Sources & Further Reading
- FDA clinical decision support software final guidance — U.S. Food and Drug Administration.
- Decision support interventions: source attributes and certified-health IT test method — U.S. Health IT Office.
- ICO artificial-intelligence audit framework: contracts and third parties — UK Information Commissioner.
Evidence checked 25 October 2026. Applicability: US and UK examples; local health law requires review. This material is general education, not legal, medical, investment or other professional advice. No vendor performance test is implied.