graphify 增量更新实战--update 与 --cluster-only 的完整工作流与源码解析【免费下载链接】graphifyTurn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.项目地址: https://gitcode.com/GitHub_Trending/graph/graphifygraphify 把代码库连同文档、SQL 模式、配置和 PDF 一起构建成可查询的知识图谱。当你持续向仓库添加或修改文件时没有必要每次都做全量重建graphify update即 skill 中的--update模式只重新抽取发生变化的文件而graphify cluster-only则跳过抽取、仅对已有图谱重新聚类。本文基于 update.md 这份 skill 参考文档完整还原增量更新的每一步命令与中间文件语义并结合 detect.py、build.py、analyze.py 的源码实现讲清楚每一步为什么这样做帮助你在自己的项目里正确运行增量更新、避免幽灵节点与陈旧边。增量更新与全量构建的分工update.md开宗明义这份参考文档只在用户显式传入--update或--cluster-only时加载首次全量构建永远不会读取它。--update适用于上一次运行之后新增或修改了文件的场景。它只重新抽取变化文件节省 token 与时间--cluster-only跳过抽取直接对现有graph.json重跑社区聚类、命名并重新生成报告。增量流程围绕一组位于graphify-out/下的中间文件展开理解这些文件的生产者—消费者关系是读懂整个 runbook 的关键中间文件写入者消费者.graphify_python安装/构建阶段所有后续命令用它定位 Python 解释器$(cat graphify-out/.graphify_python).graphify_incremental.json变更检测步骤后续 code-only 判断、合并、manifest 保存.graphify_detect.json检测步骤的派生后续 Steps 3A–6 无条件读取.graphify_extract.json抽取步骤 / 合并步骤回写Step 4 及之后.graphify_old.json合并前手动备份图 diff 展示graph.json合并与导出查询、聚类、HTML 可视化第一步用 detect_incremental 检测变更runbook 的第一条命令调用detect_incremental并与上次运行的 manifest 对比把结果写入.graphify_incremental.json$(cat graphify-out/.graphify_python) -c import sys, json from graphify.detect import detect_incremental, save_manifest from pathlib import Path result detect_incremental(Path(INPUT_PATH)) new_total result.get(new_total, 0) print(json.dumps(result, indent2, ensure_asciiFalse)) Path(graphify-out/.graphify_incremental.json).write_text(json.dumps(result, ensure_asciiFalse), encoding\utf-8\) deleted list(result.get(deleted_files, [])) if new_total 0 and not deleted: print(No files changed since last run. Nothing to update.) raise SystemExit(0) if deleted: print(f{len(deleted)} deleted file(s) to prune.) if new_total 0: print(f{new_total} new/changed file(s) to re-extract.) 其中INPUT_PATH是 skill 执行时对扫描根的占位符。命令先落盘.graphify_incremental.json再打印摘要若既没有新/改动文件、也没有删除文件则打印No files changed since last run. Nothing to update.并直接退出。从源码看detect_incremental 的返回结构正是命令中用到的那几个键在完整detect()扫描结果之上追加incremental、new_files按文件类型分组的变更子集、unchanged_files、new_total、deleted_files、excluded_files。它的判定策略是mtime 快路径 MD5 慢路径mtime 与 manifest 记录不一致 → 计算 MD5 与ast_hashkindastgraphify update用或semantic_hashkindsemanticgraphify extract用比较内容确实变了才算 changedmtime 相同但 manifest 行是在同一个文件系统时间刻度内写下的见 _mtime_may_hide_a_rewrite→ 花一次 MD5 防止同长度编辑落在同一秒被漏判其余情况走免费的 stat-only 快路径这让成熟语料库的每次--update都很便宜。另外注意deleted_files与excluded_files的区分detect.py文件从磁盘上消失才算删除对应节点是幽灵需要 prune文件仍在磁盘上但被新的 ignore 规则排除的只进入excluded_files不当作删除报告——runbook 里的deleted变量正是取自前者。填充 .graphify_detect.json变更子集与全量语料的分工检测完成后runbook 要求把结果派生为.graphify_detect.json因为 Steps 3A–6 会无条件读取该文件$(cat graphify-out/.graphify_python) -c import json from pathlib import Path r json.loads(Path(graphify-out/.graphify_incremental.json).read_text(encoding\utf-8\)) Path(graphify-out/.graphify_detect.json).write_text(json.dumps({ files: r.get(new_files, {}), all_files: r.get(files, {}), total_files: r.get(new_total, 0), total_words: r.get(total_words, 0), skipped_sensitive: r.get(skipped_sensitive, []), needs_graph: True, }, ensure_asciiFalse), encoding\utf-8\) 字段语义很明确files携带变更子集new_files驱动 Step 3A 的 AST 抽取与 Step 3B0 的缓存检查——只有真正变了的文件才进抽取管线all_files携带全量语料完整扫描的files供任何需要全库上下文的步骤使用total_files/total_words按本次增量口径取new_total/total_wordsneeds_graph: True标记本次运行需要产出图谱。敏感文件凭据、.env、私钥等在扫描阶段即被 detect.py 的正则规则静默跳过记录在skipped_sensitive里保证增量运行与全量运行的排除行为一致。纯代码变更快路径跳过语义抽取存在新文件时runbook 先检查所有变更文件是否都是代码文件$(cat graphify-out/.graphify_python) -c import json from pathlib import Path result json.loads(open(graphify-out/.graphify_incremental.json, encodingutf-8).read()) if Path(graphify-out/.graphify_incremental.json).exists() else {} code_exts {.py,.ts,.js,.go,.rs,.java,.cpp,.c,.rb,.swift,.kt,.cs,.scala,.php,.cc,.cxx,.hpp,.h,.kts,.lua,.toc,.f,.F,.f90,.F90,.f95,.F95,.f03,.F03,.f08,.F08} new_files result.get(new_files, {}) all_changed [f for files in new_files.values() for f in files] code_only all(Path(f).suffix.lower() in code_exts for f in all_changed) print(code_only:, code_only) 分支行为code_only为 True打印[graphify update] Code-only changes detected - skipping semantic extraction (no LLM needed)只运行 Step 3AAST 抽取完全跳过 Step 3B不派发任何语义子代理直接进入合并与 Steps 4–8。因为代码文件的图结构由本地确定性 AST 解析产生不需要 LLM 参与这是省 token承诺的主要来源code_only为 False任何变更文件是文档/论文/图片/视频若new_files[video]中有文件先按 transcribe.md 的 Step 2.5 把它们转写成文本再重写.graphify_detect.json把转写产物路径移入files[document]并删掉files[video]——否则原始.mp4/.mp3路径会被当作不可读媒体直接喂给语义子代理issue #1392然后正常跑完整的 Steps 3A–3C。这份code_exts白名单是 skill 侧的精简清单库侧的完整分类器 classify_file 覆盖的CODE_EXTENSIONS更广含.sql、.pas、.tf、.vue、.lisp等并把包清单文件如Cargo.toml、pyproject.toml显式路由到 AST 路径以避免重复的 file-anchored 节点。只有删除时的空抽取如果检测结果是没有新文件、只有删除runbook 要求先造一个空抽取文件让合并步骤有东西可读if [ ! -f graphify-out/.graphify_extract.json ]; then echo [graphify update] Only deletions -- creating empty extraction for merge. $(cat graphify-out/.graphify_python) -c import json from pathlib import Path Path(graphify-out/.graphify_extract.json).write_text(json.dumps({nodes:[],edges:[],hyperedges:[],input_tokens:0,output_tokens:0}), encodingutf-8) fiif [ ! -f ... ]保证已有真实抽取结果时不会被空壳覆盖。此时空的新抽取 prune_sources删除列表的组合会在合并阶段把删除文件对应的节点与边从基线图里摘掉。核心合并build_merge 与 manifest 保存合并是--update的心脏。runbook 的完整命令如下IS_DIRECTED为占位符用户给了--directed时替换为True否则False$(cat graphify-out/.graphify_python) -c import json from pathlib import Path from graphify.build import build_merge from graphify.detect import save_manifest # Load new extraction and incremental state new_extraction json.loads(Path(graphify-out/.graphify_extract.json).read_text(encoding\utf-8\)) incremental json.loads(Path(graphify-out/.graphify_incremental.json).read_text(encoding\utf-8\)) deleted list(incremental.get(deleted_files, [])) # prune_sources is ONLY for genuinely DELETED files. Changed/re-extracted files are # handled by build_merges replace-on-re-extract (#1344): every source_file in # new_chunks is dropped from the base before merge, so old/stale nodes dont survive. # Do NOT add changed here: with root passed, prune_set relativizes to the same base # as the freshly merged nodes and would DELETE the re-extracted content (#1178 is moot # now that replace — not the dedup pass — reconciles changed files). prune list(deleted) or None # Use build_merge() — reads graph.json directly without NetworkX round-trip # so edge direction (calls, implements, imports) is always preserved (#801). # Pass root so prune_sources (absolute paths from detect_incremental) are # relativized to match the graphs relative source_file values; without it # nothing is pruned and stale nodes accumulate on every update (#1361). # directedIS_DIRECTED: replace IS_DIRECTED with True if --directed was given, else # False. Without it a --directed --update silently rebuilds undirected and collapses # reciprocal A-B edges (#1392). G build_merge( [new_extraction], graph_pathgraphify-out/graph.json, prune_sourcesprune, rootINPUT_PATH, directedIS_DIRECTED, ) print(f[graphify update] Merged: {G.number_of_nodes()} nodes, {G.number_of_edges()} edges) # Write merged result back to .graphify_extract.json so Step 4 sees the full graph merged_out { nodes: [{id: n, **d} for n, d in G.nodes(dataTrue)], edges: [ # Explicit source/target last so they win over any stale attrs in d. {**{k: val for k, val in d.items() if k not in (_src, _tgt, source, target)}, source: d.get(_src, u), target: d.get(_tgt, v)} for u, v, d in G.edges(dataTrue) ], # G.graph[\hyperedges\] holds hyperedges from both existing graph.json # and new_extraction (build_merge combines them). Falling back to # new_extraction only would silently drop prior-run hyperedges (#801). hyperedges: list(G.graph.get(hyperedges, [])), input_tokens: new_extraction.get(input_tokens, 0), output_tokens: new_extraction.get(output_tokens, 0), } Path(graphify-out/.graphify_extract.json).write_text(json.dumps(merged_out, ensure_asciiFalse), encoding\utf-8\) print(f[graphify update] Merged extraction written ({len(merged_out[\nodes\])} nodes, {len(merged_out[\edges\])} edges)) # Save manifest so next --update diffs against todays state, not the # prior runs baseline (prevents ghost-node reports on subsequent updates). # root matches the build_merge call above so the manifest keys stay relative to # the scan root — portable across clones/machines, so --update keeps matching # cached files instead of missing every one after a move (#1417). # # Only stamp semantic files (docs/papers/images) that ACTUALLY produced output # THIS run (new_extraction is this runs fresh extraction, read above before the # merge overwrote the file): a changed doc whose chunk failed must stay unstamped # so the next --update re-queues it, otherwise it is marked done and its content # is lost forever (#2015). Mirrors the library extract path # (cli._stamped_manifest_files clear_semantic scan_corpus). from graphify.cli import _stamped_manifest_files _manifest_files _stamped_manifest_files(incremental[files], new_extraction, Path(INPUT_PATH)) # Changed semantic files dispatched this run but NOT stamped had their chunk fail # or be omitted; clear any stale semantic_hash so they are re-queued (#1948). _sem_types (document, paper, image) _dispatched {f for t, fl in incremental.get(new_files, {}).items() if t in _sem_types for f in fl} _stamped {f for fl in _manifest_files.values() for f in fl} _cleared _dispatched - _stamped # scan_corpus the RAW full corpus so in-root files newly excluded since last run # are dropped rather than masquerading as deletions; untouched rows preserved (#1908). _scan {f for fl in incremental[files].values() for f in fl} save_manifest(_manifest_files, rootINPUT_PATH, scan_corpus_scan, clear_semantic_cleared or None) print([graphify update] Manifest saved.) 这段命令里几乎每一处都对应 build_merge 的一个设计决策值得逐条对照源码理解1. replace-on-re-extract改动文件靠替换而非去重调和build_merge在合并前会把new_chunks中出现过的每个source_file的旧节点/旧边从基线中删除build.py 的 tier-scoped 替换逻辑所以改动文件永远不需要进prune_sources。注释特别强调prune只能放真正被删除的文件。若把改动文件塞进 pruneroot传入后 prune 集合会相对化到与新生节点相同的基路径直接把刚重建的内容删掉issue #1178 场景。替换还区分 AST 层与语义层一个文件的两个产出层可以共存AST-only 重抽不会抹掉该文件的语义超边#2333/#2336。2.root决定能不能剪掉detect_incremental返回的删除路径是绝对路径而graph.json里存的是相对source_file。传入root后prune 集合被相对化到图内同一基准才能命中不传时什么都剪不掉陈旧节点每次 update 累积#1361。源码里还有一层保险当所有 prune 条目零命中时会用绝对 prune 路径对已存相对source_file做后缀匹配反推 root 再重建 prune 集合build.py。3.directed防止有向图静默退化若不传directed一次针对有向图的--directed --update会静默按无向重建把互逆的 A↔B 边塌缩成一条#1392。build_merge的默认行为是继承磁盘上已有图的directed标志但 runbook 要求显式传值消除歧义。4. 边方向与超边不丢失注释提到build_merge直读graph.json、不经过 NetworkX 序列化往返calls/implements/imports等边方向因此总是被保留#801。回写merged_out时边序列化把source/target放在字典末尾以覆盖d中可能残留的旧属性hyperedges取自G.graph[hyperedges]——它同时包含旧图与新抽取的超边build.py 的 hyperedge carry 逻辑未被重抽、未删除的旧超边会被带入构建避免每次--update把超边集合塌缩为只剩改动文件的。5. manifest 保存只盖真正产出过的章save_manifest的三个关键参数各有出处detect.py 的文档字符串与实现_stamped_manifest_filescli.py只挑出本次实际产出了节点/超边输出的语义文件来盖章。某个改动文档的分块如果失败保持未盖章状态下次--update会重新入队否则会被标记为已完成而内容永远丢失#2015、#933clear_semantic_cleared本次派发但未盖章的文件分块失败或被省略会被清空其旧的semantic_hash防止detect_incremental(kindsemantic)误判为未变#1948scan_corpus_scan传入原始全量语料这样上次还在扫描范围、本次被新 ignore 规则排除的 in-root 文件会被从 manifest 丢弃而不是在后续运行里伪装成删除#1908root与build_merge保持一致使 manifest 键保持相对扫描根的可移植形式——移动/克隆仓库后--update仍能匹配缓存文件而不是全部失配#1417。save_manifest内部还会在序列化结果与磁盘完全一致时跳过重写避免无谓的 mtime 抖动。合并完成后runbook 指示然后像往常一样在合并图上运行 Steps 4–8聚类、命名、报告、导出等。用 graph_diff 观察本次更新改了什么Step 4 之后展示图谱差异。前提是合并之前备份旧图cp graphify-out/graph.json graphify-out/.graphify_old.json流程结束后清理rm -f graphify-out/.graphify_old.json。$(cat graphify-out/.graphify_python) -c import json from graphify.analyze import graph_diff from graphify.build import build_from_json from networkx.readwrite import json_graph import networkx as nx from pathlib import Path # Load old graph (before update) from backup written before merge old_data json.loads(Path(graphify-out/.graphify_old.json).read_text(encoding\utf-8\)) if Path(graphify-out/.graphify_old.json).exists() else None new_extract json.loads(Path(graphify-out/.graphify_extract.json).read_text(encoding\utf-8\)) G_new build_from_json(new_extract, directedIS_DIRECTED) if old_data: G_old json_graph.node_link_graph(old_data, edgeslinks) diff graph_diff(G_old, G_new) print(diff[summary]) if diff[new_nodes]: print(New nodes:, , .join(n[label] for n in diff[new_nodes][:5])) if diff[new_edges]: print(New edges:, len(diff[new_edges])) graph_diff 对比两个快照并返回new_nodes、removed_nodes、new_edges、removed_edges及一行人类可读的summary例如 3 new nodes, 5 new edges, 1 node removed。边去重键在有向图上是(u, v, relation)、无向图上是(min(u,v), max(u,v), relation)所以同一对节点之间 relation 变化会被记为一条新边加一条删除边。若没有.graphify_old.json比如首次更新diff 分支整体跳过而不报错。--cluster-only不重抽取只重聚类当图结构不变但想重新切分社区例如换了聚类参数或想刷新报告时跳过 Steps 1–3直接运行graphify cluster-only .从 cli.py 可以看到cluster-only与label走同一入口——后者等价于总是重新生成社区命名的 cluster-only。runbook 强调它自包含重新聚类、命名社区并基于现有图重新生成GRAPH_REPORT.md、graph.json和graph.html。完成后照常展示刷新后的GRAPH_REPORT.md摘要。一个重要的反面约束不要接着重跑 Steps 5–9。这些步骤读取.graphify_extract.json、.graphify_detect.json、.graphify_analysis.json等中间文件而上一次构建的收尾Step 9已经把它们清理掉了强行重跑只会得到FileNotFoundError#1392。小结graphify 的增量更新链路可以概括为manifest 变更检测mtime 快路径 MD5 校验→ 变更子集驱动的最小抽取纯代码变更跳过 LLM→ build_merge 的替换改动 剪枝删除 保留其余三分策略 → 只给真正产出过的文件盖 manifest 章 → graph_diff 验证增量效果。--cluster-only则是与之完全解耦的第二条路径只重聚类、不碰抽取中间文件。掌握这两条路径后你在大型语料库上反复迭代图谱时每次更新都只需为真正变化的部分付出成本同时不会累积幽灵节点或丢失超边与边方向。【免费下载链接】graphifyTurn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.项目地址: https://gitcode.com/GitHub_Trending/graph/graphify创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考