Presentation: From Retrieval to Reasoning: Building Production-Ready Agentic AI Systems with Knowledge Graphs

A summary of Cassie Shum's QCon Boston 2026 talk on why knowledge graphs can be a foundation for agentic systems, and the four architectural patterns she presents: context bundling, decision provenance, code as truth and agent visibility.

Context

Cassie Shum is Vice President of Ecosystem, Product Engineering at RelationalAI. The talk is about lessons learned building agentic workflows, not a claim that every system needs a knowledge graph: according to the InfoQ page, she stresses that it is not dogmatic. The conference abstract says the talk covers patterns that bridge research prototypes and enterprise deployments, including managing agent state, reliability, monitoring agent behavior and scaling.

This article summarizes the talk from the InfoQ page. It does not add benchmarks or implementation details that the speaker did not give.

Is GraphRAG Dead?

Shum opens by saying this is not a GraphRAG tutorial and not a prediction about frameworks. Her view is that retrieval has become much smarter, and the edge she saw in graph retrieval six months earlier has narrowed. Her position is that graphs are not dead, but their role is shifting from a retrieval system to the substrate of the whole system.

Why the Graph Is the Foundation

A graph database is only the underpinning. In her definition the value lies in the entities, the relationships and the logic between them, which form a semantic layer that agents and people can share. She gives several reasons to build on it:

• Long-context prompting can give good answers, but she questions repeatability, consistency and accountability across teams.
• A graph is a shared, composable source of organizational context that can be queried over time to see how relationships formed and decisions were made.
• Decisions become auditable, from the decision to the code to later changes.
• Binding several domains together exposes hidden dependencies, which she compares to a distributed monolith.
• The moat is the domain knowledge encoded in the graph, not the models, which keep changing.

Her examples come from financial institutions: which compliance regime applies, what "Tier 1" means, what was decided last quarter. A generic agent answers such questions generically, while an organizational graph can answer with specific fidelity. She also notes that institutional knowledge decays, for instance an outdated wiki rule that ends up as dead code or is reimplemented inconsistently.

The Engineering Harness

The second half of the talk is about an internal harness her team built on a knowledge graph: a deliberately minimal, Jira-like tool with ideas (epics) and tasks (stories), instead of a big, heavy tracker. She says agents shorten the feedback loop: with a bad specification a team once needed two to four weeks to notice, while agents can surface the problem in roughly 40 minutes (the InfoQ transcript is auto-generated, so treat the exact figures as approximate). In her words, agents are a fast, honest mirror of your engineering culture, and they amplify good and bad practices alike.

Pattern 1: Context Bundling

A context bundle brings together the different contexts an agent needs for a task: existing code, architecture documents and the specification, for example for an authentication task. Shum describes an experiment with two teams that had different on-call policies. She compared a knowledge-graph approach with feeding the agent all the documents (about 30 in the experiment). The implementation was correct in both cases, so accuracy was the same; the difference was efficiency, meaning how quickly the agent turns around with the answer.

In her demo, an idea to implement authentication shows the difference. An agent without organizational context would search the web and suggest OAuth or Auth0. The harness instead assembles a bundle with the team's architecture document, the GitHub repositories with existing implementations and the team's own requirements.

She also points out that feeding agents all documentation has a cost: token costs would add up across, for example, 10,000 engineers.

In her telling the value is efficiency, not accuracy, because models are good at scouring documents. Her next step is to test with thousands of documents and with a customer.

Pattern 2: Decision Provenance

Decision provenance traces how work moves from an idea to production, including the interactions between humans and agents. In Shum's example, each status change was written to the graph and summarized, which creates a feedback loop into the knowledge graph. The team could trace a merged pull request back to the original idea and to the back-and-forth with the agent before the work was picked up. She calls the result the graph artifact.

Provenance starts when a human first has an idea, not when the agent picks the task up. The paths through the graph are not linear, so the provenance has to be traversable.

Pattern 3: Code as Truth

Shum says she revised an earlier view that the specification is the real truth and code is cheap: the implementation, she argues, is where the truth of the application emerges, while the spec captures intent. She lists mistakes her team made: regenerating the spec on every pass, which erased what the builder had discovered; treating the agent as fire and forget; and assuming the engines were interchangeable. Her fix is to generate a summary of the actual implementation and store it in the graph, so that both intent and reality are modeled.

The model she proposes keeps both sides in the graph, linked by relationships: the specification as intent and the code as reality. That lets the team reconcile lineage between the two.

Pattern 4: Agent Visibility

Agent visibility answers the question of what the agents are actually doing. The queries she mentions include cost per task, tokens used, context bundle size, tasks shipped in a time window and audit history. The demo interface showed agents running on tasks; she says her team built it in-house and that it is not a product.

In another demo, a query showed how many tasks shipped in a period and what a given feature cost in tokens. A second view listed the agents running and the tasks they worked on; it ran on a managed Claude instance with Shum as the only user.

The Four Patterns at a Glance

Open Questions

• How to manage and supervise agents.
• Adaptive reasoning: when to use a graph and when not to.
• Agentic memory architecture at scale. Her current architecture is, in her words, very light.

Questions from the Audience

• Is the knowledge graph part of context engineering? Yes: it should be the context layer next to other document context, a source of truth amid context sprawl.
• What framework was the demo built on? It was built in-house as a visibility tool for her own developers. It is not a product, though the team has considered formalizing it.
• When should you use a graph rather than localized agents with specific links? Graphs carry significant overhead, so use them only when the benefit is clear. She says accuracy is no longer the main differentiator and token usage across turns is the bigger issue. Building the graph is still costly, and her company is developing skills and agents to speed that up, which requires domain expertise.
• Is there a performance penalty? It depends. Large graphs with heavy traversal may cause problems, small focused queries may not, and graph design matters.

Takeaways

• Treat context as something you assemble deliberately for each task, and measure whether it makes the agent faster, not only whether it is correct.
• Record the decisions and the human-agent interactions, so that a change can be traced back to its origin.
• Keep what the agent actually built as a first-class record next to the original intent.
• Make agent activity, cost and token usage visible to the engineering team.

Sources

• From Retrieval to Reasoning: Building Production-Ready Agentic AI Systems with Knowledge Graphs (InfoQ presentation)
• QCon Boston 2026 session page