pylingEYE N3 Reasoner

Notebooks PyPI GitHub

Pyling notebook 03

Turning model output into reviewable knowledge

A neural model can find plausible claims in text, but plausible is not the same as supported, consistent, or ready to use. Rather than hide that distinction in prompt wording, we can represent the model's output as candidate RDF facts and apply explicit N3 review policy.

This notebook follows three claims from extraction to a review queue. Every status will be another RDF statement, so downstream software can inspect both the source claim and the symbolic decision.

1. Preserve what the model actually said

Each candidate records the item asserted, the document it concerns, a confidence score, and the extraction run that produced it. Evidence requirements and a contraindication come from the surrounding knowledge graph.

Notice what is not happening: confidence is not being mistaken for truth. It is only one signal in the later policy.

from rdflib import Namespace
from pyling import reason_stream

model_output = """
@prefix : <http://example.org/ai-review#> .

:claim1 :mentions :paper42 ; :asserts :NovelMethod ; :confidence 0.91 ; :source :llmRun17 .
:claim2 :mentions :paper42 ; :asserts :DatasetB ; :confidence 0.64 ; :source :llmRun17 .
:claim3 :mentions :paper42 ; :asserts :RetractedCitation ; :confidence 0.88 ; :source :llmRun17 .

:NovelMethod :requiresEvidence :BenchmarkTable ;
               :contraindicates :RetractedCitation .
:DatasetB :requiresEvidence :DataAvailabilityStatement .
:paper42 :hasEvidence :BenchmarkTable, :DataAvailabilityStatement .
"""

2. Make the review policy explicit

The rules separate four questions:

1. Is confidence high enough to prioritize the claim?
2. Does the document contain the evidence required by the asserted item?
3. Does another claim create a known conflict?
4. Is the claim both high-confidence and supported, making it ready for a person to review?

These categories can evolve without retraining the extractor, and each derived status remains auditable.

rules = """
@prefix : <http://example.org/ai-review#> .
@prefix math: <http://www.w3.org/2000/10/swap/math#> .

{ ?claim :confidence ?score . ?score math:greaterThan 0.80 . }
=> { ?claim :reviewStatus :highConfidence . } .

{
  ?claim :mentions ?record ; :asserts ?item .
  ?item :requiresEvidence ?evidence .
  ?record :hasEvidence ?evidence .
} => { ?claim :reviewStatus :supported . } .

{
  ?firstClaim :mentions ?record ; :asserts ?firstItem .
  ?secondClaim :mentions ?record ; :asserts ?secondItem .
  ?firstItem :contraindicates ?secondItem .
} => { ?secondClaim :reviewStatus :blockedByContraindication . } .

{
  ?claim :reviewStatus :highConfidence ;
         :reviewStatus :supported .
} => { ?claim :reviewStatus :readyForHumanReview . } .
"""

3. Materialize a review queue

The reasoner combines evidence across rules until no new status can be derived. We then inspect the result as an RDFLib graph instead of parsing printed text.

result = reason_stream(
    {"sources": [rules, model_output]},
    include_input_facts_in_closure=True,
)
closure = result.as_rdflib_graph(include_input_facts=True)

AI = Namespace("http://example.org/ai-review#")
for claim in (AI.claim1, AI.claim2, AI.claim3):
    statuses = sorted(str(value).removeprefix(str(AI)) for value in closure.objects(claim, AI.reviewStatus))
    print(f"{claim.n3():<48} {', '.join(statuses)}")

<http://example.org/ai-review#claim1>            highConfidence, readyForHumanReview, supported
<http://example.org/ai-review#claim2>            supported
<http://example.org/ai-review#claim3>            blockedByContraindication, highConfidence

4. Read the decisions in context

• claim1 is high-confidence, has its required benchmark evidence, and is ready for human review.
• claim2 is supported but stays out of the priority queue because its confidence is below the chosen threshold.
• claim3 is high-confidence but blocked because it conflicts with NovelMethod. High confidence did not override a symbolic safety condition.

The rules have not declared any claim universally true or false. They created a transparent workflow decision from a particular extraction run and a particular knowledge graph.

assert (AI.claim1, AI.reviewStatus, AI.readyForHumanReview) in closure
assert (AI.claim2, AI.reviewStatus, AI.supported) in closure
assert (AI.claim3, AI.reviewStatus, AI.blockedByContraindication) in closure
assert (AI.claim3, AI.reviewStatus, AI.readyForHumanReview) not in closure
print("All review-policy expectations hold")

All review-policy expectations hold

From records to streams

So far, every notebook reasoned over one graph. Operational systems often receive an ordered series of measurements where message boundaries must remain meaningful. Notebook 04 applies reusable QUDT rules to each message independently and lets the units determine the conversion.

Python N3 reasoning in the EYE ecosystem. All notebooks