A knowledge graph over the whole datalake. Analysts get answers with their sources.
A large engineering & construction company mapped thousands of tables, procedures and KPIs into one reviewed graph, so analysts ask in plain language and every number arrives with its table, procedure and document.
- Customer
- A large engineering & construction company
- Scope
- Private cloud · enterprise datalake
- Published
What was in the way
The company’s datalake had grown over a decade of projects. Finance, procurement, project controls and plant operations each loaded their own systems into it, and the result was several thousand tables and a few thousand stored procedures, many with names that only made sense to the person who wrote them.
Analysts spent most of a request working out where a number lived and which version of a calculation was current. Two reports could disagree on the same KPI because they used different filters, and nobody could say which was right without reading the SQL behind both.
Earlier attempts at a chat interface produced fluent answers that could not be checked. The data team would not let those numbers reach a review meeting, and they were right not to.
How Agent Hive was set up
- 01
Connect read-only
The datalake was connected with a read-only service account under the existing access rules. Agents can query, never write, and every statement they run is logged with the run. - 02
Draft the graph, then review it
Agents read schemas, procedure bodies and existing reports and drafted a page per table, procedure and KPI, linked by lineage. Data owners approved pages by domain before any of them were used in answers. - 03
Validate the methods
For the questions analysts ask most, the team wrote method pages: which tables, which filters, which codes. Each was validated against real records with non-zero results before it was marked ready. - 04
Ask when the definition is open
Where the business has not settled a definition, the page carries the options and a recommended reading. The agent asks, or uses the recommendation and shows the alternative, instead of guessing silently. - 05
Cite everything, change through proposals
Every answer links the SQL it ran, the tables it read, the method page it followed and any document it opened. Corrections from analysts become proposals that an owner approves before the graph changes.
What changed
- Catalogue nodes reviewed
- 6,800+
- tables, procedures, KPIs
- Answers with sources
- 100%
- SQL, tables, method page
- Typical request
- 35 min
- was most of a day
- Writes to source systems
- 0
- read-only by design
A catalogue people can walk
The first deliverable was not an agent. It was a map. Every table, view, stored procedure and KPI in the datalake became a node, with lineage edges between them and a short page describing what each one is for. Analysts use the same graph the agents use, so when an answer cites a procedure they can open it, see what feeds it and see what it feeds.
Most of the graph was drafted by agents reading schemas, procedure bodies and existing reports. None of it went live until a person approved it. The data team reviewed pages in batches by domain, and rejected drafts were kept with the reason, which turned out to be the most useful guidance for the next batch.
Undecided definitions are asked, not guessed
“Late delivery” had two readings in the business: after the promised date, or after the requested date. Finance and procurement each used one. Rather than pick a winner, the definition page records both readings, marks one as recommended and names the owner who decides. When a question depends on it, the agent either asks which reading the analyst means or uses the recommended one and says so in the answer.
Context assembly. Context assembly for “Which vendors delivered late this month?”. 8 candidates ranked by word match, packed into a 6,000-chars window (900 reserved for context-step header + question). Late PO lines by vendor (recipe, score 0.92): in as an excerpt, 1,800 of 5,200; Late lines (old) (recipe, score 0.74): excluded, superseded; Open POs by vendor (recipe, score 0.71): in, 230; Late delivery · undecided (definition, score 0.68): in, 640; On-time delivery (kpi, score 0.60): in, 260; Delivery status codes (decode, score 0.55): in, 380; po.lines · grn.receipts (table, score 0.50): in, 420; Receipts this week (recipe, score 0.38): in, 210. The answer cites Late PO lines by vendor.
Once an owner settles a definition, the page changes through a proposal like any other edit. Earlier answers keep a link to the version they used, so a number from last month can still be explained after the definition moves.
I stopped asking where a number came from. The answer tells me, and I can open the procedure if I disagree.