SlideTwin v0.1.1
reads the slide the way a microscope does像显微镜一样阅片

Why SlideTwin

为什么需要 SlideTwin

A pathologist reads a slide by moving a field of view across the tissue at a handful of magnifications, and a report that says “no metastasis” rests on having looked at the whole node. Today's agents navigate too: CPathAgent zooms and moves like a pathologist, and GIANT pans and zooms by requesting crops. But each defines its own actions, coordinates and costs inside its own harness, where any region arrives at full resolution for free and only the final answer is scored.

病理医生阅片时，会在几个固定倍率下移动视野扫过组织，而一份写着“未见转移”的报告，前提是整个淋巴结都看过了。现在的 agent 也会导航：CPathAgent 像病理医生一样放大、移动，GIANT 通过请求裁图来平移和缩放。可它们都在各自的测试框架里自定动作、坐标和代价：任何区域一步就能免费拿到全分辨率图像，最后只给最终答案打分。

Navigation alone therefore cannot tell us whether an agent looked at the evidence it cites, searched enough tissue to support a negative, or read as a pathologist would under the same constraints. SlideTwin gives agents and people the same viewer, clock and log, so these questions have answers.

所以单凭会导航，回答不了这些问题：agent 看过它引用的证据吗？搜索的组织够支撑阴性结论吗？同样约束下，它和病理医生读得一样吗？SlideTwin 让人和 agent 共用同一个阅片器、时钟和日志，这些问题才有答案。

What SlideTwin is

SlideTwin 是什么

• One rule set for people and agents. The web UI uses the same session API as REST, MCP and the Python training interface, so a pathologist's reading and an agent's reading are made of the same actions.
• 人和 agent 同一套规则。网页界面与 REST、MCP、Python 训练接口共用同一个会话 API，病理医生和 agent 的阅片由同样的动作构成。
• Every pixel is logged. Views, thumbnails and crops are actions. The log is the reading: replaying it re-renders every image and compares hashes. In our tests, all replayed readings matched with 0 differences.
• 每个像素都有记录。视野、缩略图和裁图都是动作。日志就是阅片本身：重放时会重新渲染每张图像并比对哈希。在我们的测试中，重放的 次阅片全部一致，差异为 0。
• Physical units. Positions and distances are in micrometres from the slide's corner; time runs on a virtual clock that charges each action, as a viewer would.
• 物理单位。位置和距离以切片左上角为原点、以微米计；时间是虚拟时钟，像真实阅片器那样给每个动作计时。
• Two rule profiles. viewer gives preset magnifications, a fixed field of view and action costs. unconstrained allows free thumbnails and arbitrary crops, which is what most agent harnesses allow, so the two can be compared on the same slide.
• 两套规则档。viewer：预设倍率、固定视野、动作有代价。unconstrained：缩略图和任意裁图随取随用，相当于大多数 agent 框架的做法。两者可以在同一张切片上直接对比。
• Every common format, chosen by content: Aperio SVS, Hamamatsu NDPI, 3DHistech MRXS, DICOM WSI, Zeiss CZI, Leica SCN, Ventana BIF, Philips TIFF and iSyntax, OME-TIFF and more.
• 支持常见格式，按文件内容自动选择读取器：Aperio SVS、Hamamatsu NDPI、3DHistech MRXS、DICOM WSI、Zeiss CZI、Leica SCN、Ventana BIF、Philips TIFF 与 iSyntax、OME-TIFF 等。

Rules checked on every report

每份报告都要通过的规则

The thresholds come from the protocol file and are drafts awaiting pathologist review.

这些阈值写在协议文件里，目前是草案，有待病理医生审定。

Three ways in

三种接入方式

pip install "slidetwin[all]"
slidetwin serve --demo        # web UI, REST and MCP on http://127.0.0.1:7860

# in-process, for training: the same rules, costs and logs, no HTTP
from slidetwin.env import Env, demo_manifest
with Env(demo_manifest()) as env:
    step = env.reset("c16_29f5ca")                        # first view, slide, task, rules
    step = env.step({"action": "set_magnification", "mag": 10})
    step.image.shape, step.cost_s, step.elapsed_s          # (1024, 1280, 3), 0.8, 1.1

Why today's agent systems need a common environment

现有 agent 系统为什么需要一个统一环境

Recent systems take different routes. GIANT lets a general multimodal model pan and zoom over a slide by requesting crops, and argues that thumbnail and random-patch evaluations underestimated these models. CPathAgent navigates with zoom and move steps that mimic a pathologist. PathFinder's agents pick regions from fixed-size patches. Many evaluations of general models give them a single downsampled thumbnail. Each is a real step forward. What they share is the measuring stick, and it has gaps:

近来的系统路线各不相同：GIANT 让通用多模态模型通过请求裁图在切片上平移和缩放，并指出只用缩略图或随机小块的评测低估了这些模型；CPathAgent 以模仿病理医生的放大、移动步骤来导航；PathFinder 的 agent 从固定大小的小块中挑选区域；许多通用模型的评测只给一张降采样的缩略图。它们各有进步，但共用的那把尺子有缺口：

• Pixels are free. A harness can return any region at full resolution in one step, with no time attached. A microscope does not work that way, and behaviour learned in such a harness may not carry over to a viewer.
• 像素是免费的。测试框架一步就能返回任意区域的全分辨率图像，也不计时间。显微镜不是这样工作的，在这种框架里学到的行为未必能迁移到真实阅片器上。
• Answers are scored, evidence is not. A correct “metastasis present” counts the same whether or not the agent ever looked at the tumour.
• 只给答案打分，不看证据。一个正确的“有转移”，不管 agent 有没有看过肿瘤，得分都一样。
• Negatives come without a search. “No metastasis” is the hardest call in sentinel lymph node reading, because it needs the whole node screened; final-answer scoring rewards a guess just as much.
• 阴性结论不需要搜索。在前哨淋巴结阅片中，“未见转移”是最难下的结论，因为要筛查整个淋巴结；只按最终答案打分，猜对一样得分。
• No common units. Pixel coordinates change with the scanner and the pyramid level; a size in pixels is not a size in millimetres.
• 没有统一单位。像素坐标随扫描仪和金字塔层级变化；以像素表示的大小并不是以毫米表示的大小。
• People and agents cannot work together. They use different interfaces with different access, so a pathologist cannot step into an agent's reading, point it to a region or check its work. What the agent hands back is a list of crops or coordinates without reasons or timing: nobody can tell why it went where it went, or whether it looked at what it cites.
• 人和 agent 无法协作。两者用的界面不同、能访问的内容不同，病理医生既不能介入 agent 的阅片、给它指一个区域，也无法检查它的工作。agent 交回来的只是一串裁图或坐标，既没有理由也没有时间：没人说得清它为什么去了那里，也不知道它是否真的看过它所引用的内容。

On SlideTwin these designs become different agents in one environment, each logged, timed and checked the same way:

在 SlideTwin 上，这些设计成为同一个环境里的不同 agent，都以同样的方式被记录、计时和核验：

The first three use a plain-text protocol (one JSON action per reply), so models without tool calling can take part; the harness turns each answer into a SlideTwin report. The SlideTwin agent needs native tool calls.

前三种使用纯文本协议（每次回复一个 JSON 动作），这样不支持工具调用的模型也能参与；框架把每个答案转成 SlideTwin 报告。SlideTwin 自带 agent 需要原生工具调用。

Limits of this release

这一版的局限

• The rule thresholds and the cost model are drafts. They need pathologist review and calibration on human readings, which the pilot in the repository is set up to collect.
• 规则阈值和代价模型都是草案，需要病理医生审定，也需要用人类阅片数据校准；仓库里的试点方案就是为收集这些数据准备的。
• Reports follow one protocol, sentinel lymph node metastasis. Other tasks can already be read, timed and logged, but their answers are not checked by rules yet.
• 报告目前只遵循一个协议：前哨淋巴结转移。其他任务已经可以阅读、计时和记录，但答案还没有规则核验。
• The rules check what a report cites, not what the cited fields show: in the replay above, E1–E3 pass although the size was never measured. The log keeps what such checks would need.
• 规则核验的是报告引用了什么，而不是被引用的视野里有什么：在上面的回放里，大小从未测量过，E1–E3 却都通过了。日志里保留了做这类核验所需的全部信息。
• None of the models was trained to navigate a viewer. The environment exists so that such training, and fair comparisons after it, become possible.
• 这些模型都没有针对阅片器导航做过训练。SlideTwin 正是为了让这样的训练、以及训练后的公平比较成为可能。
• The replays on this page are single readings by one open model. They show what a log reveals, not how well models read slides.
• 本页的回放都是同一个开源模型的单次阅片，展示的是日志能揭示什么，而不是模型读片的好坏。

Get started

上手

pip install "slidetwin[all]"
slidetwin serve --demo                    # 3 public slides, 0.9 GB, http://127.0.0.1:7860
slidetwin mcp --slides DIR --manifest manifest.yaml   # MCP over stdio for a local agent
slidetwin replay --demo logs/sessions/<id>.jsonl     # replay a reading, report differences

# an open VLM through vLLM (tool calls on), read by SlideTwin's agent
vllm serve Qwen/Qwen3-VL-8B-Instruct --port 8000 --max-model-len 65536 \
  --enable-auto-tool-choice --tool-call-parser hermes --limit-mm-per-prompt '{"image": 40}'
python -m slidetwin.agents llm --backend openai --base-url http://localhost:8000/v1 \
  --model Qwen/Qwen3-VL-8B-Instruct --env http://localhost:7860 --slide cmu1_small --max-turns 16

Cite

引用

If you use SlideTwin in your work, please cite it:

如果你的工作用到了 SlideTwin，请引用：

@software{zhang2026slidetwin,
  author  = {Zhang, Wenhao and Zhou, Zhongliang and Li, Sheng},
  title   = {SlideTwin: an agent-operable whole-slide image viewer simulator},
  year    = {2026},
  version = {0.1.1},
  url     = {https://github.com/wenhaozhang0066/SlideTwin}
}

Acknowledgements

致谢

The CC0 test slides CMU-1 and CMU-1-Small-Region come from the OpenSlide test data (Carnegie Mellon University). The replays use Qwen3-VL-8B-Instruct, released by the Qwen team.

CC0 测试切片 CMU-1 与 CMU-1-Small-Region 来自 OpenSlide 测试数据（卡内基梅隆大学）。回放中使用的模型是 Qwen 团队发布的 Qwen3-VL-8B-Instruct。

References: GIANT, Buckley et al., arXiv:2511.19652 (2025). CPathAgent, arXiv:2505.20510 (NeurIPS 2025). PathFinder, Ghezloo et al., arXiv:2502.08916 (2025). QuPath, Bankhead et al., Scientific Reports 7:16878 (2017).

参考文献：GIANT，Buckley 等，arXiv:2511.19652（2025）。CPathAgent，arXiv:2505.20510（NeurIPS 2025）。PathFinder，Ghezloo 等，arXiv:2502.08916（2025）。QuPath，Bankhead 等，Scientific Reports 7:16878（2017）。