01

Goals and responsibility boundaries

项目目标与职责边界

The project is not designed merely to make an agent produce a professional-looking report. Its purpose is to make every research claim inspectable, traceable and reproducible.

项目的目标不是让 Agent 写出“看起来专业”的报告,而是让每一条科研结论都能够被检查、追踪和复现。

  1. Separate agent reasoning from direct evidence in source material区分 Agent 推理与真实材料中的直接证据
  2. Verify that cited papers exist and have the correct identity验证引用论文是否真实存在、身份是否正确
  3. Open uploaded and external PDFs and test claims against the full text打开上传 PDF 和外部论文全文,检查 Claim 是否被原文支持
  4. Attach source, page, passage, support level and reasoning to every claim为每条 Claim 提供来源、页码、证据片段、支持度和理由
  5. Preserve errors, weak evidence and download blockers instead of laundering failures保留错误、证据不足与下载阻塞,不允许为了美化报告而替换失败证据

The OpenClaw / DeepSeek agent reads the source paper, discovers candidates, proposes three ideas and decomposes claims. The evidence tracker owns paper identity, full-text retrieval, passage location, independent judgment and reporting. Search results, abstracts and agent-written context are clues only; they cannot become GREEN evidence by themselves.

OpenClaw / DeepSeek Agent 负责阅读源论文、检索候选文献、提出三个研究 Idea 并拆分 Claim;evidence-tracker 插件负责论文身份管理、真实全文下载、证据定位、独立判定与报告生成。搜索结果、摘要和 Agent 写入的 Context 只能作为线索,不能直接成为 GREEN 证据。

02

Architecture and six tools

系统架构与六个工具

A user uploads a PDF through QQBot and starts the task with a simple request. The agent locates the original upload, starts an evidence session, reads goals and limitations, searches external English papers, creates three new ideas, decomposes them into atomic claims and calls the plugin claim by claim.

用户通过 QQBot 上传 PDF,并用简单提示词启动任务。Agent 定位原始上传文件、启动 evidence session、阅读研究目标与局限、检索外部英文论文、生成三个新 Idea、拆成原子 Claim,再逐条调用插件。

  1. start_evidence_tracking: start a session and parse the source PDFstart_evidence_tracking:启动会话并解析源 PDF
  2. search_literature: discover external academic literaturesearch_literature:发现外部学术文献
  3. record_citation: record an atomic claim and source typerecord_citation:记录原子 Claim 与来源类型
  4. verify_evidence_chain: verify evidence against real full textverify_evidence_chain:核验真实全文证据
  5. generate_evidence_report: generate a transparent audit reportgenerate_evidence_report:生成透明审计报告
  6. get_session_status: inspect session statusget_session_status:检查会话状态

Discovery favors dedicated academic tools such as evidence-tracker, paper-search and scientify’s arXiv / OpenAlex capabilities. Discoverers increase candidate coverage; the tracker still makes the final independent judgment.

发现阶段优先使用 evidence-tracker、paper-search、scientify 的 arXiv / OpenAlex 等专业学术检索能力。发现器只负责扩大候选覆盖率,最终判定必须由 evidence-tracker 独立完成。

03

Ten steps in a complete task

一次完整任务的十个步骤

1. Start the evidence session

1. 启动证据链会话

start_evidence_tracking requires source_paper, source_pdf_path and idea_title. The plugin immediately parses the uploaded PDF page by page. A new session closes the previous one, clears full-text caches and repeats external searches to prevent evidence leakage across tasks.

start_evidence_tracking 必须接收 source_paper、source_pdf_path 和 idea_title。插件立即逐页解析上传 PDF;解析成功后才能继续。新 session 会关闭上一个活动会话、清理上次全文缓存,并重新执行外部检索,避免证据串线。

2. Discover external literature

2. 发现外部文献

search_literature queries arXiv, PubMed, Europe PMC, OpenAlex, Crossref, Semantic Scholar and DBLP in parallel. No result, timeout or rate limit is not proof that a paper does not exist.

search_literature 并行查询 arXiv、PubMed、Europe PMC、OpenAlex、Crossref、Semantic Scholar 和 DBLP。查询无结果、API 超时或限流都不等价于“论文不存在”。

3. Register, deduplicate and isolate identities

3. 注册、去重与身份隔离

A unified registry stores titles, authors, year, venue, DOI, arXiv ID, PMID / PMCID, database hits, candidate PDFs, download attempts, full-text SHA256 and identity conflicts. canonical_id prefers DOI, arXiv ID, PMID and PMCID before falling back to a normalized-title hash. Conflicting strong identifiers are never merged.

统一注册表保存标题、作者、年份、期刊或会议、DOI、arXiv ID、PMID / PMCID、数据库命中、候选 PDF、下载记录、全文 SHA256 和身份冲突。canonical_id 优先使用 DOI、arXiv ID、PMID、PMCID,最后才回退到规范化标题哈希。强标识符冲突时禁止合并。

4. Generate ideas and decompose claims

4. 生成 Idea 并拆分 Claim

Ideas may come from limitations, future work, failures, uncovered settings, bottlenecks, baselines, external methods, cross-domain transfer or reasonable agent inference. An idea is not evidence. Each claim must express one conclusion about one paper; compound claims are split.

Idea 可以来自源论文的 limitation、future work、失败案例、未覆盖场景、性能瓶颈、baseline、外部新方法、跨领域迁移或 Agent 的合理推理。但 Idea 本身不是证据;一条 Claim 只能表达一个结论并对应一篇论文,组合 Claim 会被拆分。

5. Record claims and source types

5. 记录 Claim 与来源类型

record_citation requires uploaded_pdf or external_literature plus cited_source, source_url, context, page_hint and section_hint. Context is only a retrieval hint. If an uploaded_pdf claim actually cites another paper, the plugin corrects the source type and downloads it. replace_citation_id may fix metadata for the same claim and paper but cannot swap claims, papers or failed outcomes.

record_citation 强制指定 uploaded_pdf 或 external_literature,并记录 cited_source、source_url、context、page_hint 和 section_hint。Context 只是检索提示。若 uploaded_pdf Claim 实际引用外部论文,插件会纠正来源类型并重新下载。replace_citation_id 只能修正同一 Claim、同一论文的元数据,不能换 Claim、换论文或洗白失败状态。

6. Verify the uploaded PDF

6. 核验上传 PDF

The plugin uses only the PDF actually parsed when the session started. It does not trust agent-supplied quotes, pages or context. It searches the whole document, sends top passages to an LLM for entailment judgment and returns real pages, passages, retrieval method, support and reasoning.

插件只使用 session 启动时真实解析的 PDF,不相信 Agent 提供的引句、页码或 Context。它在整篇文档中检索最相关片段,再交给 LLM 做语义蕴含判断,最终输出真实页码、证据片段、召回方式、支持度与理由。

7. Verify external papers

7. 核验外部论文

The tracker first confirms identity from DOI, arXiv, title and author-year signals. It then tries legal full-text paths including citation URLs, arXiv, OpenAlex, Semantic Scholar, Unpaywall, Crossref, Europe PMC / PMC, DataCite, HAL, bioRxiv / medRxiv, OpenReview, ACL Anthology, PMLR, CVF and NeurIPS. Every download must pass first-page title, DOI and arXiv checks; wrong PDFs are deleted and marked IDENTITY_CONFLICT.

插件先通过 DOI、arXiv、标题和作者年份多源确认论文身份,再尝试 citation URL、arXiv、OpenAlex、Semantic Scholar、Unpaywall、Crossref、Europe PMC / PMC、DataCite、HAL、bioRxiv / medRxiv、OpenReview、ACL Anthology、PMLR、CVF、NeurIPS 等合法全文路径。下载后强制检查首页标题、DOI 与 arXiv ID;错误 PDF 会被删除并标记 IDENTITY_CONFLICT。

8. Hybrid full-text retrieval and semantic judgment

8. 混合全文检索与语义判断

Page text is split into overlapping chunks and retrieved through BM25, lightweight hash vectors, numeric matching, page / section hints and LLM query expansion. Fused candidates are judged as ENTAILMENT, PARTIAL, CONTRADICTION or NOT_FOUND. The vector path uses lightweight hashing, not an external embedding model.

逐页文本被切成有重叠的块,再通过 BM25、轻量哈希向量、数值匹配、页码 / 章节提示和 LLM 查询扩展进行多路召回。融合后的片段被判为 ENTAILMENT、PARTIAL、CONTRADICTION 或 NOT_FOUND。这里的向量召回是轻量哈希向量,不是外部 Embedding 模型。

9. Deterministic rule review

9. 确定性规则复核

After the LLM judgment, deterministic rules downgrade unsupported proof or performance language, missing exact numbers and untested new ideas. Decimal, percentage and thousands-separated values are normalized as complete tokens, so 24.06% is no longer truncated to 06%.

LLM 判断之后还会执行通用规则:强证明或强性能语言如果没有同等强度的原文表述,GREEN 降为 YELLOW;Claim 中的精确数值没有出现在全文证据中时同样降级;未经实验验证的新 Idea 不能仅凭类比判绿。小数、百分比和千位分隔数值会作为完整 token 标准化比较,例如 24.06% 不会再被截断为 06%。

10. Emit auditable states

10. 输出可审计状态

The final report preserves successes, scientific failures and engineering blockers so readers can inspect the actual evidence instead of a polished subset.

最终报告同时保留成功、失败与工程阻塞,确保任何读者都能复查实际证据,而不是只看到被美化的结论。

04

GREEN / YELLOW / RED / BLOCKED

GREEN / YELLOW / RED / BLOCKED

  • GREEN: the real PDF directly states or demonstrates the exact claim without overstatementGREEN:实际 PDF 直接陈述或实验展示精确 Claim,且结论强度没有超过证据
  • YELLOW: the full text supports only a weaker conclusion, partial relation, analogy, transfer rationale or inferenceYELLOW:全文只支持更弱结论、部分关系、方法类比、迁移依据或合理推断
  • RED: the full text does not support the claim, contradicts it, is wrongly cited, overstated or has a false identityRED:全文不支持、与 Claim 矛盾、引用错误、严重夸大,或论文身份错误 / 不存在
  • BLOCKED: all configured retrieval paths are exhausted without auditable full text; this is an engineering state, not a scientific verdictBLOCKED:所有配置的下载与重试路径耗尽后仍无法获得可审核全文;它是工程状态,不伪装成科学判断

The report is structured Markdown containing ideas, claims, citations, source types, actual pages, passages, retrieval methods, support, states and reasons. RED and BLOCKED entries must remain unchanged.

报告默认生成结构化 Markdown,包含研究 Idea、Claim、Citation、来源类型、实际页码、证据片段、召回方式、支持度、状态与判定理由。RED 与 BLOCKED 必须原样保留。

05

Current limitations

当前限制

  1. Hybrid retrieval still relies on BM25 and lightweight hash vectors rather than scientific semantic embeddings混合检索仍以 BM25 和轻量哈希向量为主,尚未接入高质量科学语义 Embedding
  2. Scanned PDFs, complex columns, formulas and tables may reduce extraction quality扫描版 PDF、复杂双栏、公式和表格可能降低抽取质量
  3. Open-access retrieval is affected by site policy, rate limits and network conditions, so real papers may remain BLOCKED开放获取路径受站点策略、限流和网络波动影响,真实论文仍可能进入 BLOCKED
  4. LLM entailment can still be wrong, so reports must retain passages, pages and support for human reviewLLM 蕴含判断仍可能误判,因此报告必须保留实际片段、页码和支持度供人工复核
  5. Official technical pages, code repositories and datasets need their own trusted-source types and audit rules官方技术页面、代码仓库和数据集需要独立的可信来源类型与审计规则

06

Four major version changes

四次重大版本演进

v1: Metadata-validation prototype

v1:元数据验证原型

v1 registered five tools and verified paper existence mainly from DOI, arXiv and titles, then judged claims from abstracts and an LLM. It did not open uploaded PDFs, reliably inspect external full text, define source_type or manage a canonical registry and conflicts.

v1 注册五个工具,主要通过 DOI、arXiv 与标题确认论文存在,并依据摘要和 LLM 判断 Claim。它不会真正打开上传 PDF,也没有稳定的外部全文核验、明确 source_type、统一注册表或身份冲突处理,因此还不是严格的全文审计系统。

v2: Real interaction and PDF delivery

v2:打通真实交互与 PDF 交付

v2 connected the plugin to OpenClaw and QQBot, stabilized tool visibility, added Markdown → HTML / PDF, evidence tables, support values, error types and deterministic downgrades. Verification still relied on metadata, abstracts and agent context rather than real full-text review.

v2 把插件接入本地 OpenClaw 与 QQBot,修复工具不可见和调用不稳定,增加 Markdown → HTML / PDF、Evidence Table、支持度、错误类型和确定性降级规则。但核心仍依赖元数据、摘要与 Agent Context,PDF 证据并未真正全文审核。

v3: Multi-source search, Markdown reports and quality gates

v3:多源检索、Markdown 报告和质量门槛

v3 exposed search_literature as a sixth tool, expanded multi-source discovery and arXiv fallback, improved citation parsing, retries and quality gates of at least 15 citations and six external papers, and moved to polished Chinese Markdown. Uploaded evidence still depended too much on context, and external support still leaned on abstracts.

v3 将 search_literature 公开为第六个工具,扩展 Semantic Scholar、OpenAlex、Crossref、Europe PMC、DBLP 和 arXiv fallback,修复 APA / venue / arXiv 标题提取,加入多源身份查找、重试、至少 15 条 Citation 和至少 6 条外部文献的门槛,并改用高质量中文 Markdown。其局限是 PDF 内证据仍受 Agent Context 影响,外部支持关系仍主要基于摘要。

v4: A real full-text evidence audit system

v4:真实全文证据审计系统

  1. Parse the real uploaded PDF and stop trusting agent-provided pages or context强制解析真实上传 PDF,不再因 Agent 写入页码或 Context 而判绿
  2. Confirm external identity across sources and verify downloaded PDFs page by page多源确认外部论文身份,并下载真实 PDF 逐页核验
  3. Validate first-page title, DOI and arXiv and isolate IDENTITY_CONFLICT下载后检查首页标题、DOI 与 arXiv,隔离 IDENTITY_CONFLICT
  4. Unify registration and deduplication with canonical IDs用 canonical ID 统一注册与严格去重
  5. Require source_type and correct mismatched source identity强制 source_type,自动纠正来源身份
  6. Split atomic claims and prevent failure laundering by replacing claims or papers拆分原子 Claim,禁止替换 Claim / 论文来洗白失败
  7. Fuse BM25, hash vectors, numeric matching, location hints and query expansion融合 BM25、哈希向量、数值匹配、位置提示与查询扩展
  8. Treat BLOCKED as a transparent engineering state把 BLOCKED 独立为透明工程状态
  9. Govern session caches through fresh search, download and identity validation治理 session 缓存,重新检索、重新下载并重新校验
  10. Fix truncation of decimal percentages such as 24.06%修复 24.06% 等小数百分比被截断的问题

v4 upgrades the project from a multi-source abstract checker into an evidence audit system with paper identity management, real full-text retrieval, page-level passage location and transparent engineering states.

v4 把项目从“多源摘要验证器”升级为“带论文身份管理、真实全文下载、逐页证据定位和透明工程状态的证据审计系统”。

07

Acknowledgements

特别鸣谢

Thanks to Professor Yang Linyi for careful guidance and rigorous feedback, to teammate Wang Xinyu for building the project from 0 to 0.8, and to Codex and Claude Code for supporting later implementation and iteration.

感谢杨林易老师的精心指导和严厉指正;感谢队友王新宇完成项目从 0 到 0.8 的框架搭建;感谢 Codex 与 Claude Code 对后续实现与迭代的大力支持。