United States,June 12,2025–Modern healthcare runs on data. Sprawling electronic health record (EHR) databases, streaming device feeds, clinical notes dictated at the bedside, diagnostic images, and thousands of billing and quality reporting codes form the backbone of care delivery. Both care teams and administrators want to put this mountain of information to work using large language model (LLM) technology. The goal is to summarize thick chart packets in seconds, flag high-risk medication combinations before they cause adverse events, or draft a discharge summary while the physician is still with the patient.
However, the same regulations that protect private health information (PHI) now often seem to block innovation. HIPAA auditors require that data never leave a secure environment, while most commercially available AI models run on shared and opaque infrastructure that logs user inputs for later training. This creates a paralyzing dilemma. If you innovate, you risk non-compliance. If you comply, you accept manual processes that waste time, money, and clinical focus.
The author of this article is Dmitry Baraishuk, Chief Innovation Officer (CINO) at Belitsoft, a custom healthcare software development company. Healthtech businesses partner with Belitsoft to build AI-powered, complex, bespoke cloud-based, and analytics-rich platforms that comply with regulations and excel in data security.
The pitfalls of today’s workarounds
Organizations have tried two main strategies to address this dilemma, but each brings new problems. The first strategy is to run open source LLMs within the hospital’s own environment. This keeps PHI internal, but introduces new challenges, such as maintaining GPU hardware, patching security flaws, managing fresh training data, and hiring specialized engineers to tune the system at all hours. The second approach is to enforce a strict “no-AI” policy. While this eliminates certain risks on paper, it keeps clinicians stuck with manual chart review and documentation. As a result, their laptops fill with copy-and-paste tasks, web searches, and incomplete notes while patients wait.
Generic productivity tools do not help much. A standard generative AI chatbot can appear fluent on many topics, but it lacks the clinical expertise needed to interpret a blood gas panel or distinguish ICD-10 code nuances that drive reimbursement and quality metrics. Clinicians must manually translate every model suggestion, checking lab ranges, units, and code sets themselves. The model does not reduce their cognitive workload, it simply moves it.
Introducing a purpose-built middleware layer
A better path starts with a simple insight. Large language models do not need to see raw identifiers to understand clinical intent. If a secure intermediary can detect PHI, replace it with reversible placeholders, and send only the de-identified text to the model over encrypted channels, clinicians can use AI without risking exposure of sensitive data. This intermediary – called an AI privacy gateway – acts as a specialized middleware layer between EHR users and the underlying AI model.
The gateway handles four key tasks before any text leaves the secure environment:
- Automatic tokenization of PHI. Named entity recognition algorithms scan each user prompt for 18 types of identifiers, such as names, addresses, medical record numbers, and device serial numbers. Each identifier is replaced with a specific placeholder like “PATIENT_NAME_01.” The mapping table remains only within the organization’s secure environment.
- Encrypted, temporary transport. De-identified text is sent to the LLM endpoint using encrypted channels with mutual authentication. The provider’s logging systems never see the original identifiers, since they were never transmitted.
- Re-insertion on the return path. When the model responds, the gateway swaps placeholders back for the real data. Clinicians see the correct patient names, dates, and medication lists, but those details never left the secure boundary.
- Comprehensive audit and legal compliance. Every transformation, prompt, and response is hashed and time-stamped in an unchangeable log. The vendor provides a Business Associate Agreement (BAA) that binds both parties to HIPAA obligations and runs the core service on FedRAMP High certified cloud regions.
Workflow intelligence beyond privacy protection
Privacy is only the starting point. The gateway increases its value by embedding clinical intelligence into the workflow. Workflow-specific modes adjust the model’s prompts, validation checks, and reference data for each use case.
- Lab interpretation mode matches common chemistry and hematology panels, automatically normalizing units and age-specific ranges.
- Differential diagnosis mode asks follow-up questions when information is missing and cites peer-reviewed prevalence data.
- SOAP note mode organizes responses into Subjective, Objective, Assessment, and Plan sections for easy chart insertion.
- Coding assistant mode links clinical language to ICD-10, SNOMED CT, and CPT codes to help prevent billing errors.
- Drug interaction mode checks formulary and pharmacogenomics data to warn about risks like CYP450 metabolism conflicts or QT prolongation.
Clinicians can activate any mode with a single click, removing the guesswork of writing prompts.
User controls and ecosystem integration
Control features match the level of governance found elsewhere in hospitals. An optional zero-retention setting tells the gateway to delete both prompts and responses immediately after they are delivered, which is useful for highly sensitive cases. Teams working on long-term research can set retention for 30 or 90 days, with data stored in encrypted object storage within the organization’s cloud environment. Administrators manage roles so nurses, doctors, coders, and analysts each have the right level of access. An open API lets health IT teams add the gateway to EHR widgets or batch-processing systems, avoiding redundant compliance reviews for every new use case.
Multi-modal ingestion expands reach beyond typed text. Built-in speech-to-text converts bedside dictation using medical language models, and optical character recognition (OCR) turns scanned letters or faxes into searchable text. A single department can analyze echocardiography PDFs, spoken notes, and structured lab data all under the same privacy system.
Mapping mechanisms to regulatory hurdles
Healthcare leaders often ask which HIPAA requirement each feature addresses. The gateway design answers this clearly:
| Barrier | Mechanism | Outcome |
| PHI disclosure to vendors without a BAA | Automatic tokenization plus signed BAA | No raw PHI leaves the covered entity, with legal safeguards in place |
| Need for real, demonstrable protections | Unchangeable audit logs, FedRAMP High hosting | Compliance and legal teams can trace every interaction |
| Workflow disruption when context must be uploaded again | Encrypted, short-term storage tied to the session | Clinicians can pick up where they left off without re-entering data |
| Lack of clinical expertise in generic AI tools | Specialized workflow modes (lab, SOAP, coding, etc.) | Outputs are immediately useful in clinical care |
| Scalability and governance for organizations | Role-based access, seat management, open API | IT can integrate once, manage centrally, and use everywhere |
| Many types of unstructured input | Built-in speech-to-text and OCR | Scans, recordings, and photos become structured prompts |
About the Author:
Dmitry Baraishuk is a partner and Chief Innovation Officer at a software development company Belitsoft (a Noventiq company). He has been leading a department specializing in custom software development for 20 years. The department has hundreds of successful projects in AI software development, healthcare and finance IT consulting, application modernization, cloud migration, data analytics implementation, and more for startups and enterprises in the US, UK, and Canada.
Contact Details:
Country: United States
Website: https://belitsoft.com/
Email: info@belitsoft.com
Tel:+19174105757
