Why Your Healthcare AI Agent Breaks on Real Patient Records
Every clinical AI agent encounters patients with a decade of encounters, scattered across six source systems, with records that conflict and disagree with each other. Ask what medication a patient is actually on, and the answer varies widely. A patient’s tobacco history is coded in three conflicting ways. Another patient’s discontinued-and-reordered medication is coded as “stopped,” which is technically true but doesn’t reflect what actually happened.
The instinctual fix is better retrieval. Index all patient data and surface what’s relevant. That can solve for the volume of data, but not the true problem in healthcare data: multiple conflicting, time-stamped assertions across different formats and codes.
This is a post on the source of the problem, its partial solutions, and why they’re not exactly right.
The dual problem: Messiness and fragmentation
A patient record isn’t a single, consistent document. It’s a collection of facts, scattered throughout multiple sources, often repeated in different formats, and often incomplete. Data retrieval, whether it’s done by AI tools or by human healthcare professionals, usually encounters two distinct problems: messiness and fragmentation.
What is messiness?
Messiness means the same event in a patient’s medical history is coded or asserted inconsistently. It might get recorded with slightly different vocabulary, making it difficult to parse the specifics of a condition or a test result. Legacy and current code systems might read as two sources of information rather than being merged. Narrative and coded statuses don’t map cleanly to each other.
Take a simple example: a cigarette smoker. The patient’s smoking history might be coded as “Current smoker” (SNOMED 77176002) and “Cigarette Smoker” (SNOMED 65568007) in two separate entries under the same LOINC 11367-0 panel, alongside a third, differently-templated entry (LOINC 72166-2) giving the current status as “Ex-smoker” — three coded facts, two competing vocabularies, one underlying history.
What is fragmentation?
Where messiness is the same fact repeated in different ways, fragmentation is the same fact spread out across multiple records. For a human healthcare professional, that means hunting through multiple records to find the information they need. For an AI agent, that means burning through more tokens just to get a single fact straight.
Imagine a cancer patient taking pembrolizumab as part of their treatment. The start date for this drug only appears in the HPI of an oncology consultation, but the drug itself doesn’t show up in any coded medication list for four months after that start date. At best, that creates extra work for the human or the AI trying to find a clear record of medication use. At worst, it leads to potentially negative interactions with other medications.
What’s the real problem to solve?
The problem isn’t retrievability or data transfer. It’s how to create a clean, comprehensible record from multiple conflicting, time-stamped assertions. Clinical AI agents deal with fragmented data, and even answering a simple question means crawling multiple sources, over and over again. A patient’s full story spans more than just their medical history. It’s also personal context, constant updates from clinical interactions, test results, and more. With the stakes being so high if something goes wrong, clinical AI projects can quickly burn through tokens just to get the information right. But that’s not always possible.
Data standardization alone doesn’t close that gap; the data is more accessible, but there’s no function for solving messiness or fragmentation issues. AI agents need a way to adjudicate the conflicts between similar assertions, which goes beyond faster retrieval. It’s about baking in better judgment, better knowledge of how healthcare works, and ways to improve accuracy while decreasing token spend.
The partial solutions healthcare is using now
Right now, most healthcare tech uses one of the following methods to get around the data messiness and fragmentation problems.
Context stuffing
This method pushes the full patient record into an AI model’s context window, trusting the model to sort through it when it needs to generate a response to a query. It’s essentially like giving a physician a printout of a patient’s entire medical record and expecting them to find the prescription that patient was given for a specific condition and when it was given.
Why this method doesn’t always work
Giving an AI model a full record, complete with duplicates, incorrect codes, and stale information, doesn’t give it any signals as to which records it should trust, which ones it should discard, and which ones it needs to get clarification on. Not only that, but the sheer volume of information in the full record means these duplicates and erroneous records cause a sharp spike in token burn.
RAG
With RAG (retrieval-augmented generation), patient records are turned into chunks, embedded in a vector database or index, and passages are retrieved when they’re the most semantically similar to a query. Through this method, AI models can surface every place a specific fact is mentioned (e.g., a specific medication in a prescription slip, a pharmacy’s order, or a blood test), partially solving the issues caused by the pure volume of information involved.
Why this method doesn’t always work
RAG just extends an AI model’s reach. It doesn’t spontaneously get the ability to turn multiple messy records into a single authoritative data point, nor does it flag and plug gaps caused by fragmentation. RAG ranks chunks exclusively by similarity to a query; there are no checks for truth or recency. The embedding model has no representation of clinical code identity — it works on learned patterns in language, so two SNOMED codes for mutually exclusive smoking statuses can land close together in vector space purely because they use similar words, with nothing in the embedding or the nearest-neighbor search encoding that they’re competing claims about the same fact. It surfaces all relevant codes without the built-in logic to find the authoritative record. Worse, every time a clinician needs to pull that patient’s record, the search needs to happen all over again, eating up more tokens.
Chunking also flattens the relational structure that would even signal that multiple bits of data are related to the same fact. The specific wording of a query could determine whether a test result or the write-up of a primary care visit is surfaced by an AI model, without comparing the similarities between them to give a clearer picture of the situation.
Direct FHIR server queries
This method skips retrieval and queries structured fields directly, leveraging data standardization norms across healthcare systems. These queries retrieve whatever is actually stored in that field (e.g., GET /MedicationStatement?patient=123&status=active), which, if fed through an AI chatbot, can be turned into an answer a healthcare professional can actually use.
Why this method doesn’t always work
Querying a FHIR server means hitting a resource endpoint directly (e.g., GET /MedicationStatement?patient=123&status=active) and getting back exactly what’s stored for that field, correctly formatted per spec. That gets data where it needs to go, but it doesn’t resolve conflicts. And CDA compounds this: there’s no field-level query at all — a consumer fetches the whole document and parses it themselves, often finding the same clinical statement wrapped in one of several different XML shapes depending on which system generated it.
Take, for example, a provider ordering medication for a patient. They change something about an existing medication order (e.g., the dose, formulation, or frequency) before reordering. In the EHR, this shows up as “Discontinued (Reorder),” but the structured entry has a status of “stopped.” If the FHIR query just pulls up the latter, then a healthcare professional or AI has less information to go on. Both are technically valid, with no reconciliation rule telling a consumer which one to trust.
One-time cleanup or deduplication scripts
These scripts clean up patient data, merge duplicates, and normalize codes, leaving a clean snapshot of that patient’s record. The output can then be used more effectively by AI models and other systems without needing to worry about inaccurate data.
Why this method doesn’t always work
This method only works as long as new data isn’t coming in — and new data is always coming in. Take a single diagnosis split across four legacy-and-current code-system pairs into eight separate Condition resources for one patient: a cleanup script can merge these into one resource carrying both codings, as the spec allows. But the moment a new source feed lands with its own legacy coding, the same pattern reappears, unresolved, until someone reruns the script — reconciliation stays a maintenance chore instead of a property of the pipeline.
Temporal knowledge graphs
A temporal knowledge graph doesn’t just map the relationships between facts; it also tracks when each fact was true. That’s the difference between a model having an output of “the patient has condition X” and “the patient had condition X on date Y, and the condition ended on date Z.” Contrasted with an ordinary knowledge graph, this temporal layer tracks patient records more effectively, since conditions, medications, and other similar facts are rarely true forever.
Why this method doesn’t always work
Temporal knowledge graphs, while better than other knowledge graphs, are still missing an essential layer for resolving conflicts. Time alone doesn’t resolve conflict when two assertions share the same timestamp. CDA documents routinely repeat the same encounter in two places, and the document header carries the true discharge date far more reliably than the section entry does — a temporal graph would timestamp both identically, and still needs a domain-specific rule (trust the header) to know which one is right.
What none of these options solve
A patient story that stays up-to-date and resolves conflicts as new data comes in. That’s the difference between an AI model delivering an outdated version of a prescription slip based on a miscoded entry with no insight on how it compares to a different slip and delivering a snapshot of a patient’s history with a specific medication, with conflicting facts already addressed and validated against each other.