Every healthcare organization is sitting on a goldmine of unstructured clinical text -- and most of them have no idea how to extract value from it. Progress notes, discharge summaries, pathology reports, radiology reads, nursing assessments -- decades of clinical knowledge locked inside free-text fields that traditional analytics cannot touch.
Large language models change this equation fundamentally. For the first time, we have tools that can read clinical text the way a trained clinician does -- understanding context, inferring meaning, and extracting structured information at scale. But the gap between what LLMs can do and what organizations should do with them in clinical settings is vast. I have seen enough poorly implemented NLP projects to know that getting this wrong is worse than not trying at all.
The Promise and the Pitfalls
Let me be clear about what LLMs actually enable. They can take an unstructured clinical note like this:
"72 y/o male with PMHx of HTN, T2DM, and CKD stage 3b presents with 2-week history of progressive dyspnea on exertion. BNP elevated at 1,240. Echo shows EF 35%, moderate MR. Started on furosemide 40mg daily and lisinopril 5mg. Discussed goals of care."
And extract structured data: diagnoses (heart failure, diabetes, CKD), medications (furosemide, lisinopril with dosages), lab values (BNP 1,240), procedures (echocardiogram), and clinical findings (reduced EF, moderate MR). This is not magic -- it is pattern recognition at scale. But when you multiply this capability across millions of clinical notes, you unlock research and operational insights that were previously impossible.
The problem is that LLMs also do something clinicians do not: they confidently generate information that is completely wrong.
Hallucination Mitigation
The tendency of LLMs to produce plausible-sounding but incorrect output is not a bug that will be fixed in the next version. It is a fundamental property of how these models work. They predict the most likely next token based on patterns in training data -- they do not have a ground-truth database of medical facts they consult.
In a clinical context, this is dangerous. An LLM might confidently state that a patient is on metformin when the note actually mentioned metoprolol. It might infer a diagnosis that was ruled out rather than confirmed. It might fabricate a lab value that seems reasonable but was never recorded.
Effective mitigation requires multiple layers:
Retrieval-Augmented Generation (RAG) grounds LLM outputs in actual source documents. Instead of asking the model to recall information from its training, you retrieve relevant passages from the clinical note and ask the model to extract from those specific passages. This dramatically reduces hallucination by keeping the model anchored to source material.
Constrained decoding limits outputs to valid clinical concepts. If you are extracting medications, the output should be constrained to known drug names. If you are extracting ICD-10 codes, the output should be constrained to valid codes. This prevents the model from inventing concepts that do not exist.
Citation requirements force the model to identify exactly where in the source text each extracted element came from. If the model cannot point to a specific phrase that supports its extraction, the extraction should be flagged for human review.
Confidence calibration helps you understand when the model is uncertain. Well-calibrated confidence scores let you route low-confidence extractions to human review while letting high-confidence extractions flow through automated pipelines.
Validation Frameworks
I cannot stress this enough: no LLM-based extraction system should be deployed without a rigorous validation framework. The specific metrics depend on your use case, but the principles are universal.
Ground-truth annotation matters. You need a gold-standard dataset where expert clinicians have reviewed the notes and identified the correct extractions. This is expensive and time-consuming, which is why organizations skip it. Do not skip it. Your validation is only as good as your ground truth.
Measure precision and recall separately. In clinical contexts, the cost of false positives (extracting something that is not there) and false negatives (missing something that is there) are often very different. A medication reconciliation system might tolerate some false positives but need near-perfect recall. A quality measure extraction system might prioritize precision to avoid incorrectly flagging patients. Understand your error tolerance profile.
Stratify by note type and specialty. An LLM that performs well on internal medicine progress notes might fail on pathology reports or surgical operative notes. Clinical language varies dramatically across specialties and documentation contexts. Your validation must cover the full range of inputs your system will see in production.
Test edge cases explicitly. Clinical notes are full of negation ("no evidence of malignancy"), uncertainty ("cannot rule out PE"), and temporal complexity ("previously on warfarin, now on apixaban"). Build test sets that specifically probe these challenging linguistic patterns.
Monitor for drift. Clinical documentation practices change over time. New abbreviations emerge, templates evolve, and the patient population shifts. A system validated last year may not perform the same way today. Build ongoing monitoring into your production deployment.
Real-World Use Cases
With appropriate safeguards in place, LLMs can deliver substantial value across multiple clinical text applications.
Cohort Identification for Research
Clinical trials and observational studies require identifying patients who meet specific criteria -- often criteria that can only be determined by reading clinical notes. A patient with "treatment-resistant depression" or "rapidly progressive CKD" may not have an ICD code that precisely captures that clinical state, but the information is in their notes.
LLMs can screen large populations to identify potential cohort members, dramatically reducing the manual chart review burden. The key is treating LLM output as a high-recall screening step, not a final determination. Human reviewers then validate the candidates, but they are reviewing hundreds instead of thousands.
Quality Measure Extraction
Many quality measures require information buried in clinical text. Did the provider discuss smoking cessation? Was a fall risk assessment documented? Were goals of care addressed? These are yes/no questions that traditionally required manual chart abstraction.
LLMs can extract these documentation elements at scale, enabling quality measurement across entire patient populations rather than sampled audits. The regulatory and financial implications are significant -- accurate quality measurement drives reimbursement and identifies improvement opportunities.
Adverse Event Detection
Post-market drug safety surveillance depends on identifying adverse events in clinical documentation. A patient who develops new-onset rhabdomyolysis two weeks after starting a statin may not have that connection coded explicitly, but a clinician reading the note would see the temporal relationship.
LLMs can flag potential adverse events for pharmacovigilance review, surfacing signals that would be missed by structured data analysis alone. This has direct implications for patient safety and regulatory compliance.
Clinical Summarization
Patients with complex medical histories can have thousands of pages of clinical documentation. When they present for a new encounter, the receiving clinician needs a coherent summary -- not a data dump. LLMs can synthesize longitudinal clinical information into structured summaries that highlight active problems, current medications, relevant history, and recent changes.
This is higher-stakes than extraction because the model is generating new text, not just pulling out existing text. Appropriate guardrails include human review requirements, clear labeling of AI-generated content, and easy access to source documentation.
Regulatory Implications
The regulatory environment for LLM-based clinical tools is evolving rapidly, and I expect it to tighten considerably over the next two years.
If your LLM system is intended to inform clinical decisions -- whether directly or through the intermediary of extracted data -- you are likely building a medical device. The FDA's guidance on Clinical Decision Support software defines when CDS requires regulatory clearance. If your tool is intended for use by healthcare professionals, is intended for a specific clinical scenario, and provides patient-specific output, you are probably in scope.
The good news is that the FDA has shown a sophisticated understanding of AI/ML-based devices and has published guidance on predetermined change control plans that allow for model updates without re-clearance. The bad news is that documentation requirements are substantial. You need to demonstrate that your system was validated on a representative population, performs consistently across relevant subgroups, and has appropriate labeling about limitations.
My advice: engage with regulatory affairs early. Even if you believe your current application is exempt, your roadmap probably includes applications that are not. Building a quality management system and validation documentation from the start is much easier than retrofitting it later.
Where I Would Start
If I were building an LLM-based clinical text extraction capability from scratch, here is the sequence I would follow:
-
Pick a single, well-defined use case with clear success criteria and measurable business value. "Extract structured data from clinical notes" is not a use case. "Identify patients with heart failure with preserved ejection fraction from echo reports" is.
-
Build your ground-truth dataset first. Before you write a single line of model code, have clinicians annotate a representative sample of documents. This becomes your validation benchmark.
-
Start with retrieval-augmented generation. Do not ask the LLM to recall information -- give it the specific text passage and ask it to extract. This dramatically reduces hallucination.
-
Implement confidence calibration. Route low-confidence extractions to human review. Accept that not every extraction will be automated.
-
Validate rigorously before any production use. Document your validation methodology, results, and ongoing monitoring plan.
-
Build feedback loops. When human reviewers correct LLM errors, capture that signal and use it to improve the system over time.
The Bottom Line
LLMs offer a genuine step-change in our ability to extract value from clinical text. But they are tools with sharp edges, and healthcare is a domain where errors have consequences.
The organizations that succeed will be the ones that pair technological capability with methodological rigor -- not the ones that deploy the most sophisticated model, but the ones that deploy appropriately validated systems with appropriate guardrails for appropriate use cases.
The clinical text data is there. The technology is there. The question is whether your organization has the discipline to use it responsibly.