Back to Columns
AI & DX15 min read

Using Generative AI in Medical Institutions: A Practitioner's Guide to Law, Vendor Selection, and Security

July 28, 2026

Using Generative AI in Medical Institutions: A Practitioner's Guide to Law, Vendor Selection, and Security
Share this article

Using generative AI in a medical institution is fundamentally harder than in an ordinary company. Because clinical data is special care-required personal information, a data-protection dimension arises; because faulty output bears directly on patient safety, a patient-safety dimension arises; and depending on use, a regulatory (medical-device) dimension arises as well. In practice, you must design across these three layers by connecting law, guidelines, and technology.

This guide is organized for clinical, legal, and IT stakeholders: (1) the legal landscape (with article references and official guidelines), (2) how to choose a service (including contract points), and (3) how to design security.

Disclaimer: This article is general information, not legal advice. Article numbers and guideline versions change with amendments. Before any adoption decision, confirm the latest primary sources (statutes and official guidelines) and consult your legal counsel and information-security officer.

The three-layer risk structure

Three risks must be considered simultaneously:

  1. Data protection — Clinical data is "special care-required personal information" (APPI, Art. 2(3)), subject to stricter rules for acquisition, provision, and secondary use.
  2. Patient safety — Because generative AI hallucinates, unverified output bears directly on safety. Final responsibility for diagnosis and treatment rests with the physician (Medical Practitioners' Act, Art. 20: prohibition of treatment without examination).
  3. Regulatory — A program used for diagnostic/treatment decisions may qualify as a medical device (Pharmaceuticals and Medical Devices Act (PMD Act), Art. 2(4)).

The professional approach is not "use it because it's convenient," but to first classify which layer(s) a use case touches.

1. The legal landscape

1-1. APPI: entrustment vs. third-party provision

The pivotal question is whether input patient data stays within "entrustment" or amounts to "provision to a third party."

  • Entrustment (Arts. 25 and 27(5)(i)): If the provider merely processes data within the scope necessary to achieve the institution's purpose, it is entrustment—no consent is required for the provision itself, but the institution bears a duty to supervise the contractor (Art. 25).
  • Third-party provision (Art. 27): If the provider uses the data for its own purposes (typically model training/improvement), it exceeds entrustment and becomes third-party provision of special care-required data, requiring, in principle, the individual's prior consent. Opt-out third-party provision is not permitted for such data.

In June 2023 the Personal Information Protection Commission (PPC) published an alert stating that inputting personal data into a generative AI service without consent—where it is handled for purposes beyond generating the response (e.g., training)—may violate the APPI. Whether the contract guarantees "no training use of input" is therefore the dividing line.

1-2. Cross-border transfer

Many services process data in overseas regions, triggering the rules on provision to a third party in a foreign country (Art. 28). The bases for lawful transfer are broadly: the individual's consent (premised on foreign transfer); the recipient maintaining a system that meets Japan's standards; or listed-country exceptions. At adoption, always clarify "does it stay in a domestic region, and if not, what is the Art. 28 basis?"

1-3. The three-ministry guidelines

  • Institution side: MHLW "Guidelines for the Safe Management of Medical Information Systems, Ver. 7.0" (June 2026); Ver. 6.0 had restructured the document into Governance / Management / Operations volumes and Ver. 7.0 added a volume for fully outsourced server maintenance, requiring—under a risk-based approach—the three electronic-storage principles of authenticity, legibility, and preservation.
  • Provider side: METI/MIC "Guidelines for Safe Management by Providers Handling Medical Information."

Evaluate any generative AI service within this framework, documenting the accountability boundary between provider and institution.

1-4. PMD Act: SaMD classification

Under the definition of a medical device (Art. 2(4)), a program used directly for diagnostic/treatment decisions may qualify as Software as a Medical Device. MHLW's "Guidelines on Medical-Device Classification of Programs" (2021, amended 2023) turn on two elements: (1) the intended purpose as a medical device, and (2) whether malfunction could affect life or health (excluding Class I equivalents). Record-keeping and administrative uses are generally out of scope; uses that reach into clinical judgment require a classification review.

1-5. Next-Generation Medical Infrastructure Act (secondary use)

If you reuse clinical data for research or AI development, confirm this Act's framework (anonymized / pseudonymized medical information, certified operators, notice/opt-out). "Using data in care" and "reusing data" are governed by different statutes.

2. Choosing a service

Evaluate on "is there governance to keep using it safely," not on flashy features.

AspectWhat to confirm (in depth)
Data governanceIs no training use / zero data retention (ZDR) or a retention/deletion mechanism stated in a Data Processing Agreement (DPA)?
Entrustment vs. third partyIs the provider a "contractor" or a "third party"? Is no-training-use contractually guaranteed (see 1-1)?
Cross-border transferStorage/processing region; if overseas, is the Art. 28 basis (consent / standards-compliant system) settled?
Three-ministry complianceAuthenticity, legibility, preservation; documented accountability boundary
Third-party certificationISMS (ISO/IEC 27001), cloud security (ISO/IEC 27017), ISMAP, Privacy Mark
Audit & access controlTamper-evident audit logs, least privilege, authentication and key management (encryption at rest / in transit)
Change managementIs there a process to re-evaluate safety and classification on model/spec updates?
Deployment modelShared SaaS / dedicated / on-prem to match your requirements and the shared-responsibility model

Contractually, clarify the shared-responsibility model (duties differ across IaaS/PaaS/SaaS) and encode no-training-use, retention, deletion, audit rights, and incident notification into the DPA.

3. Designing security

Scope from a threat model

Assess threats and impact per use case, and put the scope in writing—use it freely for drafting/organizing, keep high-risk areas (definitive diagnosis, external input of identifiable records) out of scope.

Understand de-identification vs. legal "anonymization"

Stripping identifiers before input is effective, but it is an operational mitigation—distinct in requirements and effect from legally "anonymized information." Do not conflate operational masking with legal anonymization/pseudonymization.

Prompt injection and access boundaries

In RAG/agent architectures, indirect prompt injection via external documents or tools can make the AI read or act beyond intent. Explicitly limit what the AI can read and execute (least privilege), and isolate data per institution and per user in a multi-tenant design. Never make the AI an all-powerful administrator.

Audit-log integrity, encryption, and key management

Record "who input what into which model, when, and how it was used" in a tamper-evident form. Separate encryption at rest/in transit from key management, and control access strictly with authentication and authorization.

Continuous re-evaluation

Models and specs change. Treat operations as a cycle—narrow the scope, record, verify, review—re-evaluating safety, classification, and data handling at each guideline or spec change.

Conclusion: governance first, phased adoption

Before adopting the technology, design governance first (who does what, how far, how it is logged, and who is accountable), then expand from what can be used safely.

  1. Classify use cases by the three layers (data protection / safety / regulatory).
  2. Settle entrustment-vs-third-party, cross-border transfer, and training-use in the contract.
  3. Choose services on three-ministry compliance, third-party certification, and data residency.
  4. Design access boundaries, audit logs, encryption, and incident response.
  5. Build a clinician's final check—and re-evaluation on change—into the workflow.

Pottech develops AI-native healthcare information systems and supports ISMS certification and compliance with the three-ministry guidelines. If you want to embed generative AI safely—including regulatory compliance and security design—please get in touch.

References

This article is general information, not legal advice. Article numbers and versions may change; confirm the latest primary sources and consult experts before adoption decisions.

Share this article

Related Articles

AI & DX

AI Tools That Support Physicians: Voice Input, Summarization, Literature Search, and Chart Creation

Voice input during consultations, drafting referral letters and certificates, literature search, patient explanation materials. What AI can take on in a physician's work sits around documentation and research. We organize the division of roles between general-purpose AI and healthcare-specific AI (the AI EMR), representative tools, and the line on patient information.

September 7, 2026
AI & DX

AI Tools for Clinic Reception and Patient Contact: AI Phone, Chatbots, Web Intake, and Multilingual Support

Reception is where calls, inquiries, intake, and payment all arrive at once—and where AI's effect shows up most clearly in numbers. We cover five areas—AI phone answering, chatbots such as LINE, AI intake, translation devices and apps, and booking guidance—explaining where the burden actually falls, an adoption order that protects the patient experience, and how to handle personal information.

September 7, 2026
AI & DX

What Is Generative AI? A Plain Guide for Clinic Staff on How It Works, What It Can Do, and What to Watch

Generative AI is AI that creates new text, images, and audio. How does it differ from the AI that classifies and predicts, why did it spread so suddenly, and where does it touch clinic work? We explain the mechanism with analogies, cover the three cautions—hallucination, training use, and personal data—and point to related terms.

September 7, 2026
AI & DX

What Is an LLM? What Is Inside ChatGPT, Explained for Clinics

Inside ChatGPT, Claude, and Gemini runs an LLM—a large language model. An LLM is a mechanism that predicts the next word, and understanding that explains why it makes mistakes and what you should never type into it. We explain how it works, its limits, and what clinics should verify.

September 7, 2026
AI & DX

What Is MCP? How AI Connects to External Data and Tools

MCP (Model Context Protocol) is an open standard for connecting AI to external data and tools. It is the foundation for AI moving from answering from memory to looking things up and taking action—and it bears on how AI relates to the EMR. We explain how it works and what providers should verify.

August 11, 2026
AI & DX

What Is Software as a Medical Device (SaMD)? The Regulatory Position of AI Features

AI features in an EMR and AI supporting image diagnosis are treated entirely differently under regulation. Medical device status is determined by statutory definition, not developer intent, and qualifying products require approval or certification. We organize what SaMD is, where the line falls, and what providers should verify.

August 10, 2026
AI Karte

Explore AI Karte

An AI-native EHR connecting reception, documentation, accounting, claims, and analytics into one cycle.

View the product page

AI Karte as an Option

Most of the problems covered in this article are what AI Karte, our AI-native EHR for clinics, is built to handle. Start by seeing what it is.