Most teams building AI products in healthcare eventually face answers that sound right but can’t be traced to any sources. Was the answer hallucinated, or is there substance to it? That could be OK for other systems, such as entertainment or recommendation engines, but it's not tolerable when the answer touches a diagnosis, a medication, or a treatment plan.

Retrieval-Augmented Generation helped cut down on hallucinations by grounding answers in retrieved documents, but standard RAG still struggles with a specific kind of question, the kind that requires connecting several facts before it makes sense. Questions such as "What medications treat Alzheimer's disease but could worsen one of its known complications?" are a chain of reasoning across a diagnosis, a treatment, and a side effect. A plain vector search often misses that chain.

GraphRAG was built to address this problem. Xmartlabs' Mathias Claassen (Staff ML Engineer) and Gabriel Lema (Sr ML Engineer) walked through it in a recent webinar on applying GraphRAG to health tech. Since GraphRAG is new ground for the team, this webinar reflects their research and architectural thinking on the approach. What follows organizes what they covered into something useful for anyone deciding whether a graph belongs in their stack.

Data quality comes first

It’s tempting to treat data cleanup as something to fix later. However, an AI system is only as good as the data underneath it, and if that data is poorly selected or badly structured, no amount of prompt engineering will fix it later.

Gartner has flagged this in enterprise generative AI adoption: a large share of GenAI projects get abandoned after the proof of concept, and weak data and governance foundations are among the most commonly cited reasons.

Fixing a shaky data foundation of a running system is expensive. Teams that skip this step tend to redo the same preprocessing work several times, and each run costs more than getting it right the first time. This holds for standard RAG and for GraphRAG. A graph built on inconsistent or duplicated data doesn't recover just because it's shaped like a graph.

Why traceability matters more in healthcare

GraphRAG preserves the relationships between pieces of information instead of treating each document chunk as an isolated unit. Mathias and Gabriel framed the healthcare case around three needs.

• Connected reasoning. Clinical questions often require linking a symptom to a diagnosis, that diagnosis to a treatment, and the treatment to a possible complication, all in one query. A graph makes that chain explicit, so the model doesn't have to infer it from scattered text.
• Source traceability. Every answer can be grounded in a specific, verified medical concept, which matters when someone needs to check where a claim came from.
• Less reliance on the LLM's internal knowledge. The less an answer depends on what the model memorized during training, and the more it depends on your own validated data, the lower the risk of a confident but wrong statement.

In principle, none of this is specific to healthcare. What changes is the cost of being wrong. A hallucinated product recommendation is annoying, while a hallucinated drug interaction can cause real harm, so the traceability bar has to be higher.

What is a knowledge graph?

A GraphRAG system is built on two primitives: nodes and edges. Nodes are entities or concepts, like Alzheimer's disease, memory loss, or a specific medication. Edges connect those nodes with a labeled, directed relationship, things like "causes", "treats", or "is a symptom of". A node has a label that identifies its type (Disease, Medication, Patient) and properties that store its attributes. Edges can also have properties, including the supporting text the relationship was extracted from, which makes an answer traceable to its source.

This structure also affects retrieval speed. A relational database can represent the same information, but pulling out a multi-step relationship usually means a chain of joins that gets slower as the dataset grows. A graph database walks the connection directly. If relationships are central to your queries, that's a strong argument for this kind of storage.

Building the graph without reinventing the wheel

The biggest practical risk in building a knowledge graph isn't the tooling. It's letting the graph grow without discipline. Mathias was specific about this: start by defining an ontology with domain experts, meaning the agreed set of node labels and the relationships allowed between them. Skip this step, and an LLM extracting relationships from text will eventually invent its own inconsistent structure. The same fact might end up represented twice in opposite directions, or the same disease gets split across two duplicate nodes with its relationships divided between them.

Two starting points make this easier than building an ontology from nothing.

• FHIR. The standard structure used for medical records already models entities like Patient, Practitioner, Encounter, and MedicationRequest as connected resources. A FHIR bundle is already close to graph-shaped, making it a practical way to pull a graph directly from existing healthcare records.
• UMLS Metathesaurus. For theoretical medical knowledge that isn't tied to a specific patient, UMLS's current release defines 127 semantic types (effectively node types) and 54 relationship types. That's a pre-built vocabulary instead of one invented from scratch, and it groups different names for the same concept under one identifier, which helps prevent the duplication described above.

Once the core is in place, teams can extend it with unstructured sources, medical journals, textbooks, even Wikipedia, through an LLM-driven extraction pipeline. Chunk the text, extract candidate triplets and attributes with an LLM, then map those triplets onto the predefined ontology so nothing gets added outside the agreed structure. One approach referenced during the talk, described in the MedSumGraph paper, combines UMLS concepts with matched Wikipedia summaries and feeds both to an LLM to extract only the relationships that matter for a given term. The point is deliberately avoiding graphs so large they slow down construction and retrieval alike.

Two ways to turn a graph into an answer

Gabriel walked through two reasonable architectures rather than picking a single "correct" one.

The first depends on a well-defined graph and ontology. A natural language question comes in, the system retrieves the relevant schema, and an LLM translates the question plus that schema into a graph query language, commonly Cipher. That query executes against the graph database and returns a subgraph, which goes to an LLM along with the original question for final response generation. Because every piece of that subgraph traces back to a specific node or edge, each statement in the answer can be cited.

The second skips the strict schema dependency. It extracts candidate triplets from the question, embeds them, and compares those embeddings against pre-embedded triplets already stored in the graph, much like standard RAG's top-k retrieval. The difference shows up after that. Instead of stopping at the top matches, the system performs an n-hop subgraph expansion from the matched nodes before generating the final answer. It trades some precision for flexibility when the ontology is less rigid or still evolving.

RAG vs. GraphRAG: there isn't a universal winner

It would be convenient if GraphRAG were simply "better." It isn't. The comparison Xmartlabs walked through is a set of trade-offs across six criteria.

The honest takeaway is that GraphRAG earns its complexity when relationships and multi-hop reasoning genuinely matter for your queries, not as a default upgrade over RAG.

What the benchmarks actually show

The webinar also covered a study comparing RAG and GraphRAG on four question-answering datasets: NQ, HotpotQA, MultiHop-RAG, and NovelQA. The study adds two more strategies to the comparison. One is a classifier that picks either RAG or GraphRAG for each question. The other is an "integration" approach that combines context from both before generating the response.

Neither method wins outright. Standard RAG did better on NQ, the simpler benchmark, and GraphRAG did better on MultiHop-RAG, as expected. What did hold across all 4 benchmarks is that the integration strategy beat either method on its own. Going back to the Alzheimer's question from the intro, that would mean pulling the medication and complication relationships from the graph while a standard document retriever supplies the broader clinical context.

When a graph actually earns its place

Not every project needs this. You can use these questions to guide your decision:

1. Are relationships between entities central to your queries? If not, a vector database (unstructured text, semantic search) or a relational database (structured queries, aggregations) is probably simpler and more maintainable.
2. If relationships matter, do queries need multi-hop traversal (three or more hops)? If yes, a graph database is the right call.
3. If not, is your schema evolving rapidly? A graph's flexible schema handles this better than a rigid relational structure.
4. If the schema is stable, do you need explainability or auditability? If so, a graph's traceable paths are worth the complexity even without multi-hop needs.
5. Otherwise, simpler joins in a relational database are probably sufficient.

When your data includes both structured relationships and large volumes of unstructured documents, combining a graph with a vector store, rather than choosing one exclusively, is a legitimate answer. That echoes the benchmark result above.

Takeaways

• In healthtech, sourcing data reliably is paramount, and GraphRAG’s additional structure can help with this. Particularly for reasoning on the connection of multiple nodes is where GraphRAG shines.
• FHIR and UMLS Metathesaurus are practical starting points for a healthcare ontology rather than designing one from scratch.
• Multiple valid architectures for generating answers from a graph exist, like schema-driven text-to-query and triplet-embedding-and-expansion.
• No retrieval method wins universally. Combining RAG and GraphRAG outperformed either alone across every benchmark referenced in the webinar.
• Your decision to use a graph should follow your query patterns, not whether GraphRAG is newer or sounds more sophisticated.

Where this leaves engineering and product teams

GraphRAG asks for more upfront modeling work and more careful ontology maintenance. In exchange, it offers something standard RAG structurally cannot: answers that trace back to a specific, verifiable path through your data, and reasoning that spans multiple connected facts instead of stopping at the first similar-looking chunk. For healthcare products, where a wrong or unverifiable answer carries real consequences, that trade is often worth making.

Want the full technical walkthrough, including the live demo? You can request the complete GraphRAG for Healthtech webinar recording from Xmartlabs. Reach out to get access, or to talk through whether a knowledge graph makes sense for what you're building. If you're earlier in the AI adoption process, our AI Journey series covers how we approached rolling out AI internally across engineering, recruiting, QA, and beyond.