Clinical Document Extraction: The Open Models Guide

Every regulated organisation is sitting on a pile of documents it cannot easily read. A hospital holds discharge summaries and lab reports; a health insurer holds claim forms and prior-authorisation letters; a bank holds onboarding packs; a pharmacovigilance team holds adverse-event narratives. The information a data team needs is trapped inside prose, scans and PDFs, and the moment someone proposes sending those documents to a general-purpose model to be read, the same objection lands: this is special category or otherwise regulated data, and it is not allowed to leave the building.

That objection is why so much AI value in regulated sectors never arrives. McKinsey's State of AI puts organisational AI use at around 88 percent, with roughly a third of organisations running it in production and only about 6 percent counted as AI-mature. Stanford's AI Index 2026 finds fewer than one in ten organisations have fully scaled AI in any single function. In document-heavy regulated work, a large part of that gap is not model quality at all. It is data residency: the best available reader is a cloud service, and the data cannot go to it.

This guide defines clinical document extraction and the open, on-premise model approach to it, and anchors the category for Dilr Mira, a class of private clinical small language models from DILR.AI. It is written for the head of data science or Chief Data and AI Officer in a regulated organisation who has to turn documents into structured data without moving the documents. It covers what extraction is, why on-premise deployment matters, what a schema-as-input model does, how an open model is measured honestly, how a verifier makes the output reviewable, and how the open approach compares with general biomedical models and cloud APIs. It cedes the broader enterprise delivery question to our enterprise AI consulting guide and voice-call compliance to our work on special category data in voice calls; the subject here is documents.

This guide is shipped by the team behind Dilr Mira, a class of private clinical small language models that turn scans, lab reports and claim forms into source-grounded, schema-valid JSON on your own hardware. The commercial work of matching a model to your documents and deploying it sits inside DATS, our senior-led AI consulting practice.

What is clinical document extraction?

Clinical document extraction is the task of turning an unstructured clinical document, a scan, a lab report or a discharge summary, into structured, schema-valid data. It is not summarisation and it is not question answering. The output is a defined set of fields, each one traceable to where it came from in the source, so a person can check it. Dilr Mira is built to do exactly this task.

The distinction matters because the word "AI" hides three very different jobs. A general biomedical chat model answers medical questions in prose. A summariser compresses a document into shorter prose. An extractor reads a document and returns named fields: a document type, a patient age and sex, encounter details, vitals, labs, medications, diagnoses, procedures and allergies. Extraction is the job where structure is the point, and where a wrong or invented field is a defect rather than a stylistic choice. It is also the job most exposed to data-residency rules, because you have to feed the model the whole document, identifiers and all.

Because the output is structured, extraction can be checked mechanically in a way that free prose cannot. A returned object is either valid against the target schema or it is not. Every field either points to a span in the source or it does not. That checkability is what makes extraction a good fit for regulated work, and it is the foundation the rest of this guide builds on. When we talk about auditability later, we mean the ability to prove, field by field, that the structured output reflects the document and invented nothing.

Why does on-premise, CPU-only deployment matter for regulated health data?

On-premise deployment matters because the document itself is the sensitive asset, and most extraction options require you to send it somewhere. A claim form or a discharge summary is special category data that carries direct identifiers. If extraction happens on your own hardware, on-premises or air-gapped, the residency question disappears: no document, field or identifier leaves your perimeter, so the hardest compliance objection never has to be answered.

Dilr Mira is designed around this constraint rather than treating it as an afterthought. According to DILR.AI's published description, Mira-Q2 is open on Hugging Face under Apache-2.0, runs on CPU and takes about 2 gigabytes on disk at 4-bit quantisation, with a GPU only improving speed. That footprint is deliberate. A model small enough to run on an ordinary server, or even a laptop, can be deployed inside a hospital trust, an insurer's data centre or a bank's regulated environment without a new cloud contract, a new data-processing agreement or a residency waiver. The data-protection story becomes short because there is nothing to transfer.

The regulatory context around clinical software is worth naming precisely, because the duties bind different parties. Health data is special category data under UK GDPR Article 9, which is why residency matters so much, and the same logic governs where voice calls are processed in a regulated setting. In the UK, the DCB0129 clinical risk-management standard binds the manufacturer of health IT, while DCB0160 binds the health organisation that deploys it; the two are separate obligations on separate people. The Medicines and Healthcare products Regulatory Agency's software-as-a-medical-device regime is a topic any clinical AI team should understand, but it is context here rather than a claim: Dilr Mira's outputs are drafts for human review, not autonomous clinical decisions, so the model is an extraction tool, not a device that decides. Organisations handling NHS patient data also complete the annual NHS Data Security and Protection Toolkit self-assessment, which is far easier to satisfy when the data never moves, a point that echoes our guide to HIPAA-grade voice automation on the telephony side. On-premise is not a feature bolted onto extraction; for regulated documents, it is the precondition that lets the project start at all.

The same residency logic underpins our AI operating model consulting, which sets out where a model may run, who reviews its output and how the whole pipeline is governed, and our AI execution office, which runs that pipeline once the design is set.

What does schema-as-input mean, and what does Mira-Q2 do with a schema?

Schema-as-input means the model is told, at run time, which fields to extract, and it follows that instruction rather than a fixed built-in format. According to the published Mira-Q2 model card, the model is trained with schema-variant examples, so it follows any extraction schema injected into the system prompt, not only the clinical one it ships with. Onboarding a new document type then means supplying a schema file and a few seed examples.

This is a genuine Mira-Q2 capability today, and it is worth separating from a more advanced idea that is not yet available. The shipped model needs a schema plus a small number of seed examples to generalise to a new field set. A fully zero-shot schema, where a model handles an entirely novel schema with no examples at all, is one of the bets DILR.AI describes for the in-development Mira-3 generation, and it is not something you can rely on in the current release. Keeping that line clear matters: you can deploy schema-as-input now, with seed examples, and you should not plan around zero-shot behaviour that the live product labels as in development.

The default clinical schema gives a concrete sense of the shape. The published card lists ten required fields: a document type drawn from a fixed set such as lab report, medication list, discharge summary or progress note; a de-identified patient object holding age and sex only, never a name or medical record number; an encounter object with an ISO date and department; arrays for vitals, labs, medications, diagnoses, procedures and allergies; and a free-text extraction-notes field. Because the fields are named and typed, the output maps cleanly onto clinical data standards such as HL7 FHIR, ICD-10, SNOMED CT and LOINC, and because the patient object is de-identified by design, the extraction step does not create a new copy of the identifiers it read.

How is Mira-Q2 measured, and what do the numbers actually mean?

Mira-Q2 is measured on 782 documents across four test sets, and the results are DILR.AI's own published evaluation, rerunnable from the eval files on Hugging Face rather than asserted in a brochure. There were zero identifier leaks across all 782 documents, and field-level F1 reached 1.000 on the held-out gold set. That F1 is measured on data from the same distribution as training, and the same scorecard is honest about where the model is weaker.

The measure that travels across all four sets is JSON validity: whether the returned object is well-formed and valid against the schema. The chart below gives it by set. Two synthetic sets score 100.0 percent, test_gold held out from the training distribution and synthetic_v2 a different formatting dialect used as a robustness probe; an on-schema set of real physician documents scores 94.7 percent; and a harder set of real physician dictations spanning 39 specialties scores 85.8 percent. DILR.AI states the honest version of this plainly: Mira-Q2 scores 100 percent on training-distribution data and about 86 percent on general real physician prose, and that gap is published rather than hidden.

Reading these numbers well is the difference between a defensible pilot and an over-claim. A perfect in-distribution F1 tells you the model has learned its training generator's patterns very well; the 85.8 percent on unseen dictation tells you how far that transfers to document types it never saw, such as operative notes and physical exams. For a regulated buyer, this is the useful truth: the model is strongest on the lab, discharge and medication documents it was trained on, weaker on the long tail, and every figure is attributable to a named test set with a confidence interval. A vendor that publishes its weak set is easier to trust than one that publishes only its best, and it is the kind of evidence our writing on the proof of done for AI work argues every deployed system should carry.

How does a deterministic verifier make the output audit-ready?

A deterministic verifier makes the output checkable by testing every extraction against fixed rules before a person sees it. According to the live Dilr Mira description, every output is a draft for human review, gated by a deterministic verifier that checks schema validity, source grounding and identifier leaks. Because those checks are deterministic, they give the same verdict every time, on structured output a reviewer can inspect field by field.

The three checks map onto the three ways an extraction can go wrong. Schema validity catches a malformed or off-schema object, so a downstream system never receives a field it cannot parse. Source grounding ties each returned value to a span in the original document, so a field that the model invented rather than read is flagged rather than passed through. The identifier-leak check confirms that the structured output has not carried a direct identifier into a place it should not be, which is why the evaluation could report zero identifier leaks across all 782 documents as a measured property rather than a hope. Together they turn a probabilistic model into a pipeline whose output can be defended field by field.

The human-review step is not a caveat to apologise for; it is the design. Extraction sits upstream of clinical or financial decisions, and the model's job is to prepare structured, checked drafts, not to decide. That is why the same verifier logic underpins the redaction and minimisation thinking in our guide to a redaction-by-design pipeline: the safest system is one whose every output can be inspected, and whose failures surface at a gate instead of in production. It is also why open weights matter here. When the model is Apache-2.0 and runs on your hardware, your own team can inspect it, evaluate it and re-run its scorecard, rather than trusting a description of a service they cannot see.

How do open-weight extraction models compare with general biomedical models and cloud APIs?

Open-weight extraction models compete against two very different alternatives, and neither is a straight substitute. The first is a general biomedical model such as Meditron, BioMistral or OpenBioLLM, built to reason in prose, not to return structured fields. The second is a cloud service such as Amazon Comprehend Medical, which extracts well but processes the document off your premises. A purpose-built, on-premise extractor is a third option that fails neither test.

The general biomedical models are the sharper contrast, because they look like the obvious choice and are not. Their own authors are careful about it. The Meditron-7B model card carries an advisory notice that says, in its own words:

We recommend against deploying Meditron in medical applications without extensive use-case alignment, as well as additional testing, specifically including randomized controlled trials in real-world practice settings.

That is a strong statement from a strong model, and it is exactly right: a model trained to encode medical knowledge is not the same as a model trained to extract fields and prove where each one came from. A general biomedical model will happily produce fluent output that invents its own structure, which is the failure mode extraction cannot tolerate.

The cloud services fail a different test. Amazon Comprehend Medical is a capable, HIPAA-eligible natural-language service, but it is a cloud service, which means the document is processed off your own premises, and for special category data that is the whole objection restated. The risk of building on a cloud extraction API is not hypothetical: Google deprecated its Healthcare Natural Language API, and its own caution notice scheduled the service for shutdown on 27 May 2026, pointing users instead to a general-purpose Gemini model on Vertex AI. A purpose-built extraction endpoint was deprecated in favour of a general model in the cloud, which is the opposite of what a regulated document team needs. Where a licensed on-premise stack is genuinely appropriate, John Snow Labs is the serious incumbent; it states that it holds the number-one market rank for NLP in healthcare and life sciences for 2026, and it suits teams that want a mature, commercially supported platform and can fund the licence.

What is the best clinical document extraction model for a regulated team in 2026?

The best clinical document extraction model in 2026 depends on one question above all others: can your documents leave your premises? For a team whose data-protection impact assessment forbids sending special category data to a cloud, a small, open-weight, on-premise extractor such as Dilr Mira is the strongest fit, because it removes the transfer entirely and lets your team inspect and re-run its evaluation. That is where the open approach wins outright.

It does not win everywhere, and it would be dishonest to claim it does. A team that already has a signed data-protection agreement and an impact assessment that permits cloud processing may reasonably choose Amazon Comprehend Medical for the operational simplicity of a managed service. A large organisation that wants a mature, licensed, commercially supported platform with broad clinical coverage, and can fund it, will find John Snow Labs a credible choice. And any team whose real need is prose reasoning over the medical literature, rather than field extraction, is looking at a different tool category altogether. The honest verdict is that the on-premise open model wins on data residency, inspectability and cost of entry, and concedes on managed-service convenience and on the breadth of a mature licensed stack.

What should decide it is fit to the actual documents, not a leaderboard. The right way to choose is to take a representative sample of your own documents, define the schema you actually need, and measure validity and grounding on your data before committing. That evaluation-first discipline is the same one we apply in our writing on evaluation harness engineering, and it underpins the governance we set out in our appropriate policy document guide for special category data. It is the core of how DATS scopes any model deployment: measure on the real corpus, then talk to us about where it belongs.

What is shipped today, and what is on the roadmap?

What is shipped today is Mira-Q2, and saying so precisely matters, because the roadmap is kept separate from the release. Mira-Q2 is the current release of the Dilr Mira class, a private clinical small language model of around 3 billion parameters built on Qwen2.5-3B-Instruct, per the published specification. It is open on Hugging Face under Apache-2.0, and the model, its eval files and the full scorecard are public, so its claims can be checked rather than taken on trust.

The training is documented in the same detail. Mira-Q2 was trained with a QLoRA 4-bit adapter at rank 16 on 8,400 examples, split between gold-by-construction data grounded in real ICD-10 codes, NLM drug names and curated lab reference ranges, and schema variants for generalisation. The published training and evaluation loss were 0.132 and 0.142, a small and healthy gap that points to a model that generalised rather than memorised.

The roadmap is described honestly as in development, and it should be read that way. DILR.AI describes a Mira-3 generation with four bets: multilingual and code-switched clinical text, zero-shot schema handling, a privacy pre-patch, and a faster, smaller sibling for CPU-only fleets, alongside a trust pack of per-field statistical guarantees and signed extraction receipts. None of that is available today, and this guide treats it as direction, not product. The same applies to the wider ambition: the schema-as-input architecture is under research for other regulated document worlds, including insurance claims, prior authorisation, know-your-customer onboarding, invoices and receipts, lab networks and legal intake. Those are research directions, not shipped products, and the commercial route to any of them runs through our consulting practice rather than a self-serve purchase.

The limitations are published alongside the capability, which is the point. Mira-Q2 is English only. It was trained on synthetic data, so real clinical retraining with a design partner improves accuracy on a given document mix. Its validity on general real documents across 39 specialties is about 86 percent, strongest on the lab, discharge and medication types it was trained on. And every output is a draft for human review, never an autonomous clinical decision. A regulated buyer should read that list as a feature: a model whose maker publishes the weak set and the honest gap is a model you can put in front of an auditor. To see the numbers yourself, read the category page for our strategy writing and the model's own scorecard.

Can Dilr Mira make clinical decisions on its own?

No. Dilr Mira is an extraction tool, and according to its live description every output is a draft for human review, never an autonomous clinical decision. Its job is to turn a document into structured, source-grounded fields that a clinician or analyst then checks. It does not diagnose, prescribe or triage, and it is not presented as a medical device. Keeping the model firmly upstream of any decision is what makes it safe to deploy in a regulated setting.

Does my data leave my infrastructure with Dilr Mira?

No. According to the live Dilr Mira description, the model runs entirely on your own hardware, on-premises or even air-gapped, and can run on CPU alone, so no document, field or identifier is sent to any third-party API. That is the central design choice: the data never leaves the building. It is what lets a regulated organisation extract structured data from special category documents without a cloud transfer, a residency waiver or a new data-processing agreement.

Want to go deeper? See Mira-Q2 on Hugging Face and its open scorecard, read our enterprise AI consulting guide, see the wider AI solutions we build, book an AI placement diagnostic, or read about our approach to placing AI where the data actually lives.

Written by the Dilr.ai engineering team, practitioners who ship enterprise AI in production. Follow us on LinkedIn for shipping notes, or subscribe via the RSS feed.