Most tools stop at summaries or code stubs. Verigraph closes the full loop: extract claims from papers, run methods in a sandbox, and write VALIDATES or REFUTES verdicts back into a live Neo4j knowledge graph.
Paper → Claim → Method → Code → Sandbox Run → Result → Graph Update
A focused view of three real papers: claims conflict, a sandbox run executes, and measured evidence writes VALIDATES / REFUTES back to the graph.
The shared graph contains six papers, 19 claims, and a growing archive of reproducible runs. The guided path follows the Adam, Wilson, and AdamW disagreement.
vis-network shows papers, claims, methods, citations, and CONTRADICTS edges. Evidence tab: all claims say "no runs yet".
Click the violet ◆ method node. Console tab loads the runnable spec with optional parameter overrides.
The production app executes the selected method in a fresh Daytona sandbox, applies your parameter changes, and persists the measured result.
A ⚡ Run node appears. Claims update to VALIDATES or REFUTES with run ids and exact metrics in the evidence cards.
RocketRide queries Neo4j via NL→Cypher. With Cognee enabled, semantic recall over paper text and run logs augments answers — e.g. Wilson's GD 0.000 vs Adam 0.425.
Not a mockup: each feature maps to a real module in app/ with verified smoke tests.
Upload via arXiv URL (HTML or PDF with retry/fallback), PDF drop, or raw text. LLM extracts claims, methods, datasets, citations, and cross-paper relations into structured JSON.
Idempotent MERGE into Neo4j Aura. Papers, claims, methods, runs, and artifacts as first-class nodes. CONTRADICTS edges surface real disagreements between papers.
Any method node becomes a runnable single-file experiment. Curated implementations for demo papers; live LLM codegen for newly uploaded papers via Grok.
Daytona isolated sandboxes when DAYTONA_API_KEY is set. Local subprocess fallback keeps the demo unkillable. Captures stdout, stderr, duration, exit code.
Writes Run + Artifact nodes back to Neo4j with IMPLEMENTS, VALIDATES, and REFUTES edges. Failed runs are evidence too; the graph remembers what broke.
Live claim table with validated / refuted / untested filters, plus an Insights tab that adjudicates CONTRADICTS pairs from sandbox verdicts and tracks coverage.
vis-network visualization with search, deep links, keyboard shortcuts, export, zoom controls, and tabbed side panel (Evidence · Insights · Console · Ask).
Every Ask now recalls relevant Cognee Brain context before the graph-grounded answer. Visitor sessions remain isolated by anonymous id—no email or IP is sent to Cognee.
One-click pristine state: clears Run/Artifact nodes, reloads papers, wipes local run dirs. Evidence returns to "no runs yet" for a clean stage demo.
Papers and measured runs persist in Butterbase outside Neo4j, preserving an auditable evidence archive alongside fresh cloud execution.
CONTRADICTS edges between claims from different papers. Insights ranks each conflict as untested, leading, resolved, or both-supported once runs exist.
Download JSON snapshots or a shareable markdown evidence brief (print to PDF). Compare two runs side-by-side and batch-run never-executed methods.
Track how claim verdicts flip as new runs land. Citation layout mode highlights CITES edges across papers.
When DAYTONA_API_KEY is set on Butterbase, RUN executes live sandboxes. Named workspace snapshots persist per demo email beyond the access gate.
Neo4j holds the evidence graph — claims, conflicts, runs, verdicts. Cognee adds active semantic memory over 25 indexed documents, so every Ask can reason across paper prose and experiment history before returning graph-grounded evidence.
POST /api/ask calls Cognee recall for every question. Ask "what did Wilson show about Adam generalization?" and retrieve GD 0.000 vs Adam 0.425 before the graph answer.scripts/sync_cognee.py merges local documents with the complete Butterbase run archive, then writes the corpus to managed Cognee Brain.Neo4j answers "which claim does run X validate?" Cognee answers "what did the paper say about test error?" Together they make the Ask tab feel like talking to a lab notebook, not a database console.
| Neo4j Aura | Structured evidence: Paper, Claim, Method, Run, VALIDATES, REFUTES, CONTRADICTS |
|---|---|
| Cognee | Active cloud memory: 25 indexed documents, every-Ask recall, visitor-isolated sessions |
| Enable (local) | COGNEE_ENABLED=true + ROCKETRIDE_GATEWAY_* |
| Enable (cloud) | COGNEE_CLOUD=true + COGNEE_SERVICE_URL + COGNEE_API_KEY → Sessions on platform.cognee.ai |
Eight Python modules, one RocketRide agent pipeline, five sponsor integrations, and active Cognee semantic memory—each step verified independently.
Paper text → claims, methods, datasets, citations, claim_relations as JSON. Schema-validated; fallback method if LLM returns none.
Idempotent MERGE into Neo4j Aura. Label-scoped deletes on shared instance. CONTRADICTS, CITES, FROM, DESCRIBED_IN edges.
Method node → single-file experiment. Output contract: last stdout line is JSON with metrics and claim_checks verdicts.
Execute in Daytona sandbox (auto | daytona | local). Persist run record to runs/ with full capture.
Write Run + Artifact nodes. IMPLEMENTS → Method, VALIDATES/REFUTES → Claim, PRODUCED → Artifact (stdout capped 4k).
Canned Cypher: claims, conflicts, methods, evidence. Powers the evidence table and CLI inspection.
Every question recalls semantic context from Cognee Brain, then combines it with authoritative graph evidence.
Sync papers and runs to Butterbase paper2result app. Archive outside the graph; gateway also powers all LLM calls.
Full implementation details for HackWithBay judges and developers who want to reproduce or extend the system.
| RocketRide | Local engine on :5565. Pipeline verigraph.pipe: agent_rocketride (6 waves, internal memory) + db_neo4j (NL→Cypher) + tool_python. LLM routed through one gateway node. |
|---|---|
| Neo4j Aura | Cloud graph database (neo4j+s://). Product is the graph, 8 node labels, 10+ relationship types. Label-scoped ops on shared Aura instance. |
| Butterbase | AI gateway (OpenAI-compatible) for all LLM calls: extraction, codegen, agent. Dedicated paper2result app for papers + run history persistence. |
| Daytona | Isolated sandbox execution via daytona-sdk. --backend auto selects Daytona when API key present; local subprocess fallback otherwise. |
| Cognee | Managed semantic memory — 25 documents in Brain, recall() on every Ask, and anonymous visitor-specific Sessions. Neo4j remains authoritative for verdict edges. |
| Gateway | Butterbase AI gateway: ROCKETRIDE_GATEWAY_BASE_URL + ROCKETRIDE_GATEWAY_KEY |
|---|---|
| Model | x-ai/grok-4.3 (Grok) via ROCKETRIDE_GATEWAY_MODEL. Single route for extract, codegen, and agent pipeline. |
| Temperature | 0.0 for extraction/codegen (deterministic JSON). Agent uses pipeline instructions with accuracy rule for metric quoting. |
| Direct key | XAI_API_KEY available for direct xAI access if needed; production path uses Butterbase gateway. |
| GET / | Landing page (marketing) |
|---|---|
| GET /demo | Live demo UI (vis-network graph + side panel) |
| GET /api/graph | Full node/edge payload for graph visualization |
| GET /api/evidence | Evidence table rows (claim → verdict or "no runs yet") |
| POST /api/run/{method_id} | Closed loop: codegen → execute → curate. Body: {"params":{}}. Query: backend=auto|local|daytona |
| POST /api/ask | Question → Cognee recall (when enabled) + RocketRide agent pipeline → graph-grounded answer |
| POST /api/memory/recall | Direct semantic memory search over indexed papers and run logs |
| POST /api/upload-arxiv | Fetch arXiv (HTML/PDF) → extract → load graph |
| POST /api/upload-file | PDF or text file → extract → load graph |
| POST /api/reset | Clear runs, reload papers, pristine evidence state |
| Setup | python3 -m venv .venv && pip install -r requirements.txt && cp .env.example .env |
|---|---|
| Load graph | .venv/bin/python app/graph.py --reset |
| Smoke tests | scripts/check_neo4j.py · scripts/check_pipeline.py |
| Start UI | .venv/bin/uvicorn app.server:app --port 8787 → http://localhost:8787 |
| CLI loop | .venv/bin/python scripts/demo_loop.py wilson2017-m1 |
| Reset demo | .venv/bin/python scripts/reset_demo.py or UI reset button |
| Sync Cognee | COGNEE_ENABLED=true .venv/bin/python scripts/sync_cognee.py — backfill papers + runs |
Click the violet method node, watch the code run in a sandbox, and see the evidence table flip live: GD 0.000 vs Adam 0.425, digit-for-digit.
Open the demo ▶