Using Generative AI in Medical Institutions: A Practitioner's Guide to Law, Vendor Selection, and Security

Using generative AI in a medical institution is fundamentally harder than in an ordinary company. Because clinical data is special care-required personal information, a data-protection dimension arises; because faulty output bears directly on patient safety, a patient-safety dimension arises; and depending on use, a regulatory (medical-device) dimension arises as well. In practice, you must design across these three layers by connecting law, guidelines, and technology.

This guide is organized for clinical, legal, and IT stakeholders: (1) the legal landscape (with article references and official guidelines), (2) how to choose a service (including contract points), and (3) how to design security.

Disclaimer: This article is general information, not legal advice. Article numbers and guideline versions change with amendments. Before any adoption decision, confirm the latest primary sources (statutes and official guidelines) and consult your legal counsel and information-security officer.

The three-layer risk structure

Three risks must be considered simultaneously:

1. Data protection — Clinical data is "special care-required personal information" (APPI, Art. 2(3)), subject to stricter rules for acquisition, provision, and secondary use.
2. Patient safety — Because generative AI hallucinates, unverified output bears directly on safety. Final responsibility for diagnosis and treatment rests with the physician (Medical Practitioners' Act, Art. 20: prohibition of treatment without examination).
3. Regulatory — A program used for diagnostic/treatment decisions may qualify as a medical device (Pharmaceuticals and Medical Devices Act (PMD Act), Art. 2(4)).

The professional approach is not "use it because it's convenient," but to first classify which layer(s) a use case touches.

1. The legal landscape

1-1. APPI: entrustment vs. third-party provision

The pivotal question is whether input patient data stays within "entrustment" or amounts to "provision to a third party."

• Entrustment (Arts. 25 and 27(5)(i)): If the provider merely processes data within the scope necessary to achieve the institution's purpose, it is entrustment—no consent is required for the provision itself, but the institution bears a duty to supervise the contractor (Art. 25).
• Third-party provision (Art. 27): If the provider uses the data for its own purposes (typically model training/improvement), it exceeds entrustment and becomes third-party provision of special care-required data, requiring, in principle, the individual's prior consent. Opt-out third-party provision is not permitted for such data.

In June 2023 the Personal Information Protection Commission (PPC) published an alert stating that inputting personal data into a generative AI service without consent—where it is handled for purposes beyond generating the response (e.g., training)—may violate the APPI. Whether the contract guarantees "no training use of input" is therefore the dividing line.

1-2. Cross-border transfer

Many services process data in overseas regions, triggering the rules on provision to a third party in a foreign country (Art. 28). The bases for lawful transfer are broadly: the individual's consent (premised on foreign transfer); the recipient maintaining a system that meets Japan's standards; or listed-country exceptions. At adoption, always clarify "does it stay in a domestic region, and if not, what is the Art. 28 basis?"

1-3. The three-ministry guidelines

• Institution side: MHLW "Guidelines for the Safe Management of Medical Information Systems, Ver. 6.0" (May 2023), restructured into Governance / Management / Operations volumes, requiring—under a risk-based approach—the three electronic-storage principles of authenticity, legibility, and preservation.
• Provider side: METI/MIC "Guidelines for Safe Management by Providers Handling Medical Information."

Evaluate any generative AI service within this framework, documenting the accountability boundary between provider and institution.

1-4. PMD Act: SaMD classification

Under the definition of a medical device (Art. 2(4)), a program used directly for diagnostic/treatment decisions may qualify as Software as a Medical Device. MHLW's "Guidelines on Medical-Device Classification of Programs" (2021, amended 2023) turn on two elements: (1) the intended purpose as a medical device, and (2) whether malfunction could affect life or health (excluding Class I equivalents). Record-keeping and administrative uses are generally out of scope; uses that reach into clinical judgment require a classification review.

1-5. Next-Generation Medical Infrastructure Act (secondary use)

If you reuse clinical data for research or AI development, confirm this Act's framework (anonymized / pseudonymized medical information, certified operators, notice/opt-out). "Using data in care" and "reusing data" are governed by different statutes.

2. Choosing a service

Evaluate on "is there governance to keep using it safely," not on flashy features.

Contractually, clarify the shared-responsibility model (duties differ across IaaS/PaaS/SaaS) and encode no-training-use, retention, deletion, audit rights, and incident notification into the DPA.

3. Designing security

Scope from a threat model

Assess threats and impact per use case, and put the scope in writing—use it freely for drafting/organizing, keep high-risk areas (definitive diagnosis, external input of identifiable records) out of scope.

Understand de-identification vs. legal "anonymization"

Stripping identifiers before input is effective, but it is an operational mitigation—distinct in requirements and effect from legally "anonymized information." Do not conflate operational masking with legal anonymization/pseudonymization.

Prompt injection and access boundaries

In RAG/agent architectures, indirect prompt injection via external documents or tools can make the AI read or act beyond intent. Explicitly limit what the AI can read and execute (least privilege), and isolate data per institution and per user in a multi-tenant design. Never make the AI an all-powerful administrator.

Audit-log integrity, encryption, and key management

Record "who input what into which model, when, and how it was used" in a tamper-evident form. Separate encryption at rest/in transit from key management, and control access strictly with authentication and authorization.

Continuous re-evaluation

Models and specs change. Treat operations as a cycle—narrow the scope, record, verify, review—re-evaluating safety, classification, and data handling at each guideline or spec change.

Conclusion: governance first, phased adoption

Before adopting the technology, design governance first (who does what, how far, how it is logged, and who is accountable), then expand from what can be used safely.

1. Classify use cases by the three layers (data protection / safety / regulatory).
2. Settle entrustment-vs-third-party, cross-border transfer, and training-use in the contract.
3. Choose services on three-ministry compliance, third-party certification, and data residency.
4. Design access boundaries, audit logs, encryption, and incident response.
5. Build a clinician's final check—and re-evaluation on change—into the workflow.

Pottech develops AI-native healthcare information systems and supports ISMS certification and compliance with the three-ministry guidelines. If you want to embed generative AI safely—including regulatory compliance and security design—please get in touch.

References

• PPC, "Alert on the Use of Generative AI Services (June 2, 2023)"
• PPC, "Guidelines on the APPI (General Rules)" / "(Provision to a Third Party in a Foreign Country)"
• MHLW, "Guidelines for the Safe Management of Medical Information Systems, Ver. 6.0 (Overview)"
• MHLW, "About Medical-Device Programs" / "Guidelines on Medical-Device Classification of Programs"

This article is general information, not legal advice. Article numbers and versions may change; confirm the latest primary sources and consult experts before adoption decisions.