KG-PROVENANCE produced results beyond the state of the art by shifting the focus from extracting isolated facts from text to updating a specific, existing knowledge graph. Conventional information-extraction systems generally identify entities and relations without considering what is already present in the graph, whether an entity is new, or whether an existing statement has become outdated. The project formalised text-driven knowledge graph updating as five operations: confirming existing knowledge, adding a relation between known entities, creating and connecting an emerging entity, inferring links required to integrate that entity into the graph, and deprecating outdated knowledge.
The central result is EMERGE, a benchmark that connects evolving Wikipedia passages with corresponding Wikidata updates over time. Its construction pipeline combines deterministic comparison of successive knowledge graph snapshots, temporal alignment with changes in text, large-language-model-assisted filtering, and manual validation. Unlike static information-extraction datasets, EMERGE evaluates whether a system produces the correct operation for the state of the graph at a particular time. It also distinguishes semantically plausible textual predictions from executable updates expressed using the graph’s canonical entities and relations.
A targeted knowledge-graph-aware experiment provided an important result beyond the original evaluation. The same large language model was tested with no graph context, with realistically retrieved graph context, and with the correct graph context supplied directly. Providing the correct context substantially improved near-exact and executable updating. Exact executable accuracy reached 50.3% for confirming existing knowledge, 39.6% for adding knowledge and 36.9% for deprecating outdated knowledge. However, performance fell considerably when the required context had to be retrieved automatically. Adding irrelevant retrieved information could also reduce performance, even when the correct context was present. Furthermore, the model remained far from perfect when given the correct entities, relations and existing statements.
These findings show that the difficulty of updating knowledge graphs is not simply a matter of converting names into graph identifiers. Two separate research challenges remain: retrieving and canonicalising the relevant graph context, and reasoning jointly over that context and the textual evidence to produce the correct update. The benchmark therefore provides both an evaluation resource and a diagnostic framework for developing future knowledge-graph-aware systems.
The results can contribute to more transparent and evidence-based maintenance of knowledge resources. Linking updates to supporting text can help researchers, developers and domain experts inspect why information is present, identify outdated statements and validate proposed changes. This is particularly relevant in rapidly evolving fields such as biomedicine, where newly published evidence must be integrated into existing resources while retaining human oversight. Publicly available datasets, code and evaluation tools support reuse, comparison and extension by other research groups.
Further uptake will require dedicated architectures that jointly retrieve, reason over and update knowledge graphs. Additional research is also needed to include dates, quantities and other literal attributes; extend the approach to languages beyond English and to specialised knowledge graphs; and address longer delays between changes in textual and structured sources. Demonstration in operational biomedical and industrial settings will be necessary to assess reliability, integration costs and the role of expert verification. Continued maintenance of provider-independent evaluation tools, reproducible local baselines and interoperable representations of textual evidence will also support long-term uptake. The principal current exploitation route is open and non-commercial, while commercial adoption would require application-specific integration, validation and support.
