Resume-Matcher Prompt 工作流设计本地偏差检测与重试机制全解析【免费下载链接】Resume-MatcherThe #1 AI Harness for Building Resumes, PDFs, Cover Letters more, locally with 100 LLMs support.项目地址: https://gitcode.com/GitHub_Trending/re/Resume-Matcher本文以docs/agent/architecture/prompt-workflow-design.md为骨架结合apps/backend中 improver、refiner、llm、enrichment 的实际源码与测试系统讲解 Resume-Matcher 简历优化管线如何检测 LLM 输出的内容偏差丢条目、伪造技能、字数膨胀、改名公司并通过重试与反馈闭环提升输出质量。读完本文你将掌握偏差评估器的五类检查设计、硬/软严重级重试策略、动态反馈后缀构造方法以及该方案与当前 diff-based 改进管线的演进关系。1. 背景管线缺什么1.1 现状管线Resume-Matcher 的简历优化管线由多轮 LLM 调用与本地处理串联而成设计文档给出了当时的骨架extract_keywords() ─→ improve_resume() ─→ 6 safety nets ─→ refine_resume() ─→ diff aux LLM #1 LLM #2 (local) LLM #3 LLM #4-6extract_keywords()LLM #1 从职位描述JD抽取关键词improve_resume()LLM #2 依据 JD 与关键词改写整份简历6 道安全网位于app/routers/resumes.py本地修补 LLM 输出refine_resume()LLM #3 做关键词注入、AI 味短语清除与主简历对齐校验后续 LLM 调用生成封面信、外联消息、面试准备等辅助内容。1.2 已有的传输层重试llm.py:complete_json管线底层经由complete_json()apps/backend/app/llm.py调用 LLM传输层已经具备结构层面的重试能力Malformed JSON → 以 Output ONLY valid JSON 提示重试输出截断空数组→ 以 Output COMPLETE JSON 提示重试空响应 → 重试每次重试温度递增0.1 → 0.3 → 0.5 → 0.7。源码佐证llm.py中_get_retry_temperature()维护temperatures [base_temp, 0.3, 0.5, 0.7]并按模型能力决定是否传 temperature_supports_temperature()查询 LiteLLM 模型注册表且对 Anthropic Opus 4.x 禁用、对 Moonshot kimi-k2.6 固定为 1.0。1.3 缺失的内容质量重试传输层重试只解决结构坏了不解决内容错了。文档明确指出四个缺口不检查 LLM 输出是否忠实于原简历不检查条目数量是否被保留工作经历、教育、项目不检查字数是否失控膨胀过度润色不检查新技能是否来自 JD 而非凭空捏造对内容偏差没有重试机制——只有结构失败才重试。1.4 偏差在哪伤人偏差类型当前检测何时被捕获代价丢失工作条目无用户在预览中发现信任流失伪造技能validate_master_alignmentrefinerimprove 之后、refine 期间被静默移除浪费一次 LLM 调用字数翻倍无永远不会简历显得 AI 味过重personalInfo 出现在输出中_preserve_personal_info修补improve 之后被修补而非被预防日期丢失_restore_original_dates修补improve 之后被修补而非被预防关键结论resumes.py中的安全网是事后打补丁不会反馈给 LLM 让它产出更好的结果refiner 捕获伪造内容时选择移除而不是让 LLM 重试。这意味着所有纠错都是被动的LLM 的调用成本照付但产出质量没有随迭代提升。2. 总体设计本地偏差评估器 重试循环2.1 架构在improve_resume()与安全网之间插入一个本地评估器。若评估器发现偏差超过阈值则携带具体反馈重试 LLM 调用最多重试 1 次总共 2 次尝试避免延迟爆炸┌─────────────────────┐ │ retry with feedback │ │ (max 1 retry) │ └────────┬────────────┘ │ ▼ extract_keywords ──→ improve_resume ──→ evaluate_deviation ──→ safety nets ──→ refine ──→ done LLM #1 LLM #2 (local, free) (local) LLM #3 ▲ │ │ fail │ pass └────────────────┘2.2 评估器本地、零 LLM 成本评估器纯本地运行——不调用 LLM。它把 improve 输出与原简历数据逐项对比返回 pass/fail 与具体反馈文本。五类检查全部是字符串/集合比较唯一可能新增的 LLM 成本仅发生在确实检测到偏差并重试这一条路径上。2.3 各组件落位组件文件函数偏差评估器app/services/improver.pyevaluate_deviation()重试循环app/services/improver.pyimprove_resume()内部偏差反馈提示app/prompts/templates.pyDEVIATION_FEEDBACK_SUFFIX富化并行化app/routers/enrichment.pygenerate_enhancements()自动策略选择app/services/improver.pyauto_select_strategy()3. Change 1偏差评估器的五类检查3.1 检查一章节数量保持LLM 不得删减或新增工作经历、教育、项目条目。这是文档点名的最常见失败模式输出变长时LLM 会静默丢掉最早的工作经历条目而用户往往到打印 PDF 时才察觉。def _check_section_counts( original: dict[str, Any], improved: dict[str, Any], ) - list[str]: issues [] for key, label in [ (workExperience, work experience), (education, education), (personalProjects, project), ]: orig_count len(original.get(key, [])) imp_count len(improved.get(key, [])) if imp_count orig_count: issues.append( fDropped {orig_count - imp_count} {label} fentries ({orig_count} → {imp_count}). fKeep ALL original entries. ) elif imp_count orig_count: issues.append( fAdded {imp_count - orig_count} new {label} fentries ({orig_count} → {imp_count}). fDo NOT invent new entries. ) return issues注意该检查是双向的不仅拦截删减imp_count orig_count也拦截凭空新增条目imp_count orig_count。3.2 检查二字数比improve 后的简历字数不应超过原稿 1.8 倍。过度润色会让简历一眼看上去就是 AI 生成的。def _check_word_count( original: dict[str, Any], improved: dict[str, Any], max_ratio: float 1.8, ) - list[str]: orig_words _count_description_words(original) imp_words _count_description_words(improved) if orig_words 0 and imp_words orig_words * max_ratio: return [ fDescription word count increased {imp_words / orig_words:.1f}x f({orig_words} → {imp_words}). fKeep descriptions concise — do not over-elaborate. ] return []为什么是 1.8xfull策略合法地扩充要点2x 以上则几乎必然是过度润色。1.8x 为 full 策略留出余量同时能抓住失控膨胀。源码佐证improver.py中已有_count_description_words()统计 workExperience、personalProjects 的 description 数组与 summary 的字数且verify_diff_result()的第 4 项检查同样使用 1.8 倍阈值result_words orig_words * 1.8可见该阈值在 diff 管线中已作为告警标准落地。3.3 检查三伪造技能检测输出中的新技能必须来自 JD 关键词而非 LLM 凭空发明。def _check_fabricated_skills( original: dict[str, Any], improved: dict[str, Any], job_keywords: dict[str, Any], ) - list[str]: orig_skills _extract_skills_set(original) imp_skills _extract_skills_set(improved) jd_skills _extract_jd_skills_set(job_keywords) fabricated imp_skills - orig_skills - jd_skills if fabricated: # Only flag if more than 1 — single-word variants are common if len(fabricated) 1: sample , .join(sorted(fabricated)[:5]) return [ fAdded skills not in original resume or job description: f{sample}. Only use skills from the original resume or JD. ] return []为什么这能抓到 refiner 漏掉的validate_master_alignment()apps/backend/app/services/refiner.py是对照主简历校验——而主简历的技能可能比本次定向输入更多因此伪造技能有可能绕过它。此检查更严格技能既不在原简历、也不在 JD 中才会被标记。为降低误报仅当伪造项多于 1 个时才触发单词的变体很常见如 Python vs Python 3。3.4 检查四公司与职位保持LLM 不得改名公司或职位头衔。def _check_entry_identity( original: dict[str, Any], improved: dict[str, Any], ) - list[str]: issues [] for key, title_field, subtitle_field, label in [ (workExperience, title, company, work experience), (education, degree, institution, education), ]: orig_entries original.get(key, []) imp_entries improved.get(key, []) for i, (orig, imp) in enumerate(zip(orig_entries, imp_entries)): orig_id ( orig.get(subtitle_field, ).strip().lower() ) imp_id ( imp.get(subtitle_field, ).strip().lower() ) if orig_id and imp_id and orig_id ! imp_id: issues.append( fChanged {label}[{i}] {subtitle_field} from f{orig.get(subtitle_field)} to f{imp.get(subtitle_field)}. fNever change company names, institutions, or degrees. ) return issues该检查使用小写化.lower()做不区分大小写的身份比较避免把 Acme Corp → acme corp 这类无害大小写变化误判为改名。3.5 检查五personalInfo 泄漏improve 提示词明确要求 LLM 跳过 personalInfo该字段会从原简历保留。如果它出现在输出中就是一次偏差。def _check_personal_info_leak(improved: dict[str, Any]) - list[str]: pi improved.get(personalInfo) if pi and isinstance(pi, dict) and any(pi.values()): return [ Output includes personalInfo — the improve prompt excludes it. Do NOT include personalInfo in output. ] return []源码佐证apps/backend/app/prompts/templates.py中IMPROVE_SCHEMA_EXAMPLE明确不含 personalInfo三个策略提示词都写了 Do NOT include personalInfo in your output - it will be preserved from the original resumeimprover.py的_check_for_truncation()注释也说明 personalInfo 被有意排除由_preserve_personal_info()从原简历恢复。检查五是这一约定的最后一环。3.6 组合评估器dataclass class DeviationResult: passed: bool issues: list[str] severity: str # none | soft | hard def evaluate_deviation( original: dict[str, Any], improved: dict[str, Any], job_keywords: dict[str, Any], ) - DeviationResult: Local quality gate — zero LLM cost. Returns pass/fail with specific feedback for retry prompt. issues: list[str] [] # Hard failures — always retry issues.extend(_check_section_counts(original, improved)) issues.extend(_check_entry_identity(original, improved)) hard_issues len(issues) # Soft failures — retry only on first attempt issues.extend(_check_word_count(original, improved)) issues.extend(_check_fabricated_skills(original, improved, job_keywords)) issues.extend(_check_personal_info_leak(improved)) if not issues: return DeviationResult(passedTrue, issues[], severitynone) severity hard if hard_issues 0 else soft return DeviationResult(passedFalse, issuesissues, severityseverity)严重级划分是重试策略的核心硬失败章节数量、条目身份→ 总是重试软失败字数、技能、personalInfo→ 仅记录不强制重试refiner 反正会兜底处理伪造技能。4. Change 2improve_resume()中的重试循环在improve_resume()外部包一层最多 2 次的尝试循环第 1 次原始调用 第 2 次携带反馈的重试async def improve_resume( original_resume: str, job_description: str, job_keywords: dict[str, Any], language: str en, prompt_id: str | None None, original_resume_data: dict[str, Any] | None None, ) - dict[str, Any]: # ... existing setup (lines 170-211) stays the same ... max_attempts 2 last_result None for attempt in range(max_attempts): current_prompt prompt if attempt 0 and last_result is not None: # Append deviation feedback to prompt evaluation evaluate_deviation( original_resume_data or {}, last_result, job_keywords, ) feedback \n.join(f- {issue} for issue in evaluation.issues) current_prompt ( prompt f\n\n--- CRITICAL CORRECTIONS (your previous output had these problems) ---\n feedback \n\nFix ALL issues listed above. Do not repeat these mistakes. ) logger.info( Deviation retry (attempt %d/%d): %s, attempt 1, max_attempts, feedback, ) result await complete_json( promptcurrent_prompt, system_promptYou are an expert resume editor. Output only valid JSON., max_tokens8192, ) _check_for_truncation(result) validated ResumeData.model_validate(result) result_dict validated.model_dump() # Evaluate deviation (skip on last attempt — take what we get) if attempt max_attempts - 1 and original_resume_data: evaluation evaluate_deviation( original_resume_data, result_dict, job_keywords, ) if evaluation.passed: return result_dict # Only retry on hard failures (dropped entries, renamed companies) # Soft failures (word count, skills) proceed with warning if evaluation.severity soft: logger.warning( Soft deviation detected (not retrying): %s, evaluation.issues, ) return result_dict # Hard failure — retry logger.warning( Hard deviation detected (retrying): %s, evaluation.issues, ) last_result result_dict continue return result_dict # Should not reach here, but safety fallback return result_dict4.1 关键设计决策最多 1 次重试共 2 次尝试重试是叠加延迟的。一次 improve 调用耗时 8–12s重试一次再加 8–12s而整个流程本身已有 240s 超时预算加太多重试会吃掉安全余量。硬/软严重级分流丢失条目与改名公司总是重试字数与技能问题只产生警告——伪造技能反正会被 refiner 处理。反馈进提示词重试把具体问题追加到原提示词后面LLM 能精确看到自己哪里错了。这比泛泛的 try again 有效得多。评估器在安全网之前运行resumes.py的安全网_preserve_personal_info、_restore_original_dates等是事后打补丁的。评估器必须作用于 LLM 的原始输出打补丁之前因此它住在improve_resume()内部而不是 router 层。评估零 LLM 成本5 项检查全是本地字符串/集合比较唯一的额外 LLM 成本是检测到偏差时的重试调用本身。4.2 与传输层重试的协作关系注意区分两个层次llm.py:complete_json()的重试针对结构故障JSON 解析失败、截断、空响应本方案的重试针对内容偏差结构合法但内容失实。两者互补传输层保证格式对评估器保证内容对。设计文档也明确llm.py不在改动范围内。5. Change 3偏差反馈提示后缀不需要新增提示词模板。反馈由评估器的 issues 列表动态构造作为后缀追加到既有提示词上保持提示词系统精简、无新增模板需要维护。后缀格式--- CRITICAL CORRECTIONS (your previous output had these problems) --- - Dropped 1 work experience entry (4 → 3). Keep ALL original entries. - Changed workExperience[0] company from Acme Corp to ACME Corporation. Never change company names. Fix ALL issues listed above. Do not repeat these mistakes.每个 issue 一行- {issue}用\n连接落点即重试循环中current_prompt prompt 后缀 收尾指令的拼接逻辑。6. Change 4generate_enhancements()并行化app/routers/enrichment.py的富化端点原来对每个条目串行调用complete_jsonfor item_id, answers in answers_by_item.items(): # ... build prompt ... result await complete_json(prompt) # blocks on each item enhancements.append(...)改为asyncio.gather并行async def _enhance_single_item(item_id, item, answers, questions_by_id): Generate enhanced descriptions for a single item. # ... build prompt (existing code) ... result await complete_json(prompt) additional_bullets result.get(additional_bullets, []) if not additional_bullets: additional_bullets result.get(enhanced_description, []) if not isinstance(additional_bullets, list): additional_bullets [] additional_bullets [str(b) for b in additional_bullets if b] return EnhancedDescription( item_iditem_id, item_typeitem.get(item_type, experience), titleitem.get(title, ), original_descriptionitem.get(current_description, []), enhanced_descriptionadditional_bullets, ) # In generate_enhancements(): tasks [ _enhance_single_item(item_id, item_details.get(item_id, {}), answers, questions_by_id) for item_id, answers in answers_by_item.items() if item_details.get(item_id) ] results await asyncio.gather(*tasks, return_exceptionsTrue) enhancements [] for result in results: if isinstance(result, Exception): logger.warning(fFailed to enhance item: {result}) else: enhancements.append(result)影响若用户有 4 个条目待富化延迟从约 40s4 次串行调用降到约 10s1 批并行。从当前源码看apps/backend/app/routers/enrichment.py的generate_enhancements()仍是逐条for循环调用complete_json说明并行化仍是待落地的优化提案不过regenerate_items端点已经使用了asyncio.gather(*tasks, return_exceptionsTrue)模式每个条目一个协程、异常隔离、失败只记录不中断可作为并行化落地的同仓库先例。设计文档同时建议用return_exceptionsTrue保证单条目失败不拖垮整批。7. Change 5自动策略选择新增auto_select_strategy()按关键词匹配率在nudge/keywords/full之间选择供prompt_idauto时使用def auto_select_strategy( original_resume_data: dict[str, Any], job_keywords: dict[str, Any], ) - str: Select tailoring strategy based on keyword match percentage. Uses the same keyword matching logic as the refiner to determine how much work the LLM needs to do. from app.services.refiner import calculate_keyword_match match_pct calculate_keyword_match(original_resume_data, job_keywords) if match_pct 70: return nudge # already close — light rephrasing elif match_pct 35: return keywords # relevant but missing key terms else: return full # needs significant restructuring集成点在improve_resume()selected_prompt_id prompt_id or DEFAULT_IMPROVE_PROMPT_ID if selected_prompt_id auto and original_resume_data: selected_prompt_id auto_select_strategy(original_resume_data, job_keywords) logger.info(Auto-selected strategy: %s, selected_prompt_id)这需要在app/prompts/templates.py的IMPROVE_PROMPT_OPTIONS列表中加入auto前端才能提供该选项。源码佐证templates.py中IMPROVE_PROMPT_OPTIONS目前只含 nudge / keywords / full 三项DEFAULT_IMPROVE_PROMPT_ID keywordscalculate_keyword_match()已在app/services/refiner.py中实现test_refiner.py中TestCalculateKeywordMatch验证了 0–100 的百分比语义与词边界匹配如 Go 不应匹配 Google。策略阈值语义≥70% 说明简历与 JD 已经很接近轻改写即可35%–70% 说明相关但缺关键术语35% 说明需要整体重构。8. 变更总览、不变量与延迟影响8.1 全部变更#内容位置类型LLM 成本1evaluate_deviation()— 5 项本地检查app/services/improver.py新函数02improve_resume()中的重试循环app/services/improver.py修改既有仅硬失败时 1 次调用3偏差反馈后缀动态构造improve_resume()内无新模板04generate_enhancements()并行化app/routers/enrichment.py修改既有05auto_select_strategy()app/services/improver.py新函数08.2 明确不动的部分llm.py—— 传输层重试与 JSON 质量检查保持不变refiner.py—— 对齐校验、AI 短语移除、关键词注入保持不变resumes.py安全网 ——_preserve_personal_info、_restore_original_dates等保持不变不新增依赖不用 LangChain不引入工作流引擎不新增提示词模板 —— 反馈动态构造。8.3 延迟影响场景现在改动后快乐路径无偏差~12s~12s评估器增加 1ms硬偏差丢条目~12s坏输出~24s重试产出正确输出富化4 个条目~40s~10s自动策略N/A1ms本地关键词匹配9. 实施顺序evaluate_deviation() 重试循环—— 影响最大先拦最严重的失败富化并行化—— 直接了当立刻拿到延迟收益自动策略选择—— 小改动但需要前端支持 auto 选项。10. 设计演进从全量输出偏差检测到 diff-based 管线本文依据的设计文档状态标注为Superseded——其方案已被 diff-based improvement design 取代。理解这段演进对读者非常重要因为它解释了同一批偏差问题在现仓库中如何被结构性地解决核心思路转变不再让 LLM 输出整份简历全量输出 每个字段都是幻觉机会而是只输出想改什么的定向 diff。原简历结构程序化保留偏差问题从源头消除丢条目/改名公司/日期截断/personalInfo 泄漏/customSections 伪造 →在 applier 中被按构造消除apply_diffs()的路径白名单_ALLOWED_PATH_PATTERNS与黑名单_BLOCKED_PATH_PREFIXES、_BLOCKED_FIELD_NAMES直接禁止触碰 identity 字段伪造技能/发明指标 →显式 diff 本地校验器捕获。现仓库apps/backend/app/services/improver.py的实际管线是generate_resume_diffs() ──→ apply_diffs() ──→ verify_diff_result() ──→ refine_resume() LLM 本地 本地 LLM其中verify_diff_result()就是本设计文档思想在现管线中的落点。它实现了与evaluate_deviation()高度重合的本地质量检查apps/backend/app/services/improver.py无变更应用时告警No changes were applied章节数量保持workExperience / education / personalProjects 计数一致身份字段不变company、title、institution、degree 逐项比对字数比不超过 1.8x发明指标检测\d%、\dx、$\d正则新增数字若不在原文中则告警。测试佐证见 test_verify_diffs.py覆盖丢弃工作经历/教育/项目条目告警改名公司/头衔/院校告警字数翻倍告警新增 40% / $5M 指标告警正常增长不告警等用例第 5 项发明指标检测正是本设计文档检查三/检查四之外额外补上的新维度。在编排侧app/routers/resumes.py的_improve_preview_flow()现在依次调用generate_resume_diffs → apply_diffs → verify_diff_result随后仍执行安全网_preserve_personal_info、_restore_original_dates、restore_dates_from_markdown、_preserve_original_skills、_protect_custom_sections与多轮 refinement——即文档中6 道安全网 refine在 diff 时代作为纵深防御保留了下来。值得注意设计文档中 Change 4富化并行化与 Change 5auto 策略并不属于 diff 重构的范围两者至今仍是未落地的优化提案generate_enhancements()仍串行、IMPROVE_PROMPT_OPTIONS仍无 auto任何希望贡献的开发者都可以从这里入手。11. 总结这份设计文档回答了一个贯穿 Resume-Matcher 优化管线始终的问题如何让 LLM 输出结构正确且内容忠实。它给出的方案——本地零成本评估器 硬/软严重级重试 动态反馈后缀——虽然在后续演进中被 diff-based 方案以从源头消除偏差的方式超越但其检查维度章节数量、字数比、伪造技能、身份字段、personalInfo 泄漏全部沉淀为现仓库verify_diff_result()的本地校验逻辑并被测试固化为回归保障。而富化并行化与自动策略选择两个提案至今仍有明确的落地空间。对想深入源码的读者建议按以下路径阅读管线编排apps/backend/app/routers/resumes.py的_improve_preview_flow()本地校验apps/backend/app/services/improver.py的apply_diffs()、verify_diff_result()与generate_resume_diffs()传输层重试与温度策略apps/backend/app/llm.py的complete_json()、_get_retry_temperature()、_supports_temperature()对齐校验与关键词匹配apps/backend/app/services/refiner.py的validate_master_alignment()、calculate_keyword_match()测试回归apps/backend/tests/unit/test_verify_diffs.py、apps/backend/tests/unit/test_refiner.py后续设计docs/superpowers/specs/2026-03-23-diff-based-improvement-design.md【免费下载链接】Resume-MatcherThe #1 AI Harness for Building Resumes, PDFs, Cover Letters more, locally with 100 LLMs support.项目地址: https://gitcode.com/GitHub_Trending/re/Resume-Matcher创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考 Resume-Matcher Prompt 工作流设计：本地偏差检测与重试机制全解析
Resume-Matcher Prompt 工作流设计本地偏差检测与重试机制全解析【免费下载链接】Resume-MatcherThe #1 AI Harness for Building Resumes, PDFs, Cover Letters more, locally with 100 LLMs support.项目地址: https://gitcode.com/GitHub_Trending/re/Resume-Matcher本文以docs/agent/architecture/prompt-workflow-design.md为骨架结合apps/backend中 improver、refiner、llm、enrichment 的实际源码与测试系统讲解 Resume-Matcher 简历优化管线如何检测 LLM 输出的内容偏差丢条目、伪造技能、字数膨胀、改名公司并通过重试与反馈闭环提升输出质量。读完本文你将掌握偏差评估器的五类检查设计、硬/软严重级重试策略、动态反馈后缀构造方法以及该方案与当前 diff-based 改进管线的演进关系。1. 背景管线缺什么1.1 现状管线Resume-Matcher 的简历优化管线由多轮 LLM 调用与本地处理串联而成设计文档给出了当时的骨架extract_keywords() ─→ improve_resume() ─→ 6 safety nets ─→ refine_resume() ─→ diff aux LLM #1 LLM #2 (local) LLM #3 LLM #4-6extract_keywords()LLM #1 从职位描述JD抽取关键词improve_resume()LLM #2 依据 JD 与关键词改写整份简历6 道安全网位于app/routers/resumes.py本地修补 LLM 输出refine_resume()LLM #3 做关键词注入、AI 味短语清除与主简历对齐校验后续 LLM 调用生成封面信、外联消息、面试准备等辅助内容。1.2 已有的传输层重试llm.py:complete_json管线底层经由complete_json()apps/backend/app/llm.py调用 LLM传输层已经具备结构层面的重试能力Malformed JSON → 以 Output ONLY valid JSON 提示重试输出截断空数组→ 以 Output COMPLETE JSON 提示重试空响应 → 重试每次重试温度递增0.1 → 0.3 → 0.5 → 0.7。源码佐证llm.py中_get_retry_temperature()维护temperatures [base_temp, 0.3, 0.5, 0.7]并按模型能力决定是否传 temperature_supports_temperature()查询 LiteLLM 模型注册表且对 Anthropic Opus 4.x 禁用、对 Moonshot kimi-k2.6 固定为 1.0。1.3 缺失的内容质量重试传输层重试只解决结构坏了不解决内容错了。文档明确指出四个缺口不检查 LLM 输出是否忠实于原简历不检查条目数量是否被保留工作经历、教育、项目不检查字数是否失控膨胀过度润色不检查新技能是否来自 JD 而非凭空捏造对内容偏差没有重试机制——只有结构失败才重试。1.4 偏差在哪伤人偏差类型当前检测何时被捕获代价丢失工作条目无用户在预览中发现信任流失伪造技能validate_master_alignmentrefinerimprove 之后、refine 期间被静默移除浪费一次 LLM 调用字数翻倍无永远不会简历显得 AI 味过重personalInfo 出现在输出中_preserve_personal_info修补improve 之后被修补而非被预防日期丢失_restore_original_dates修补improve 之后被修补而非被预防关键结论resumes.py中的安全网是事后打补丁不会反馈给 LLM 让它产出更好的结果refiner 捕获伪造内容时选择移除而不是让 LLM 重试。这意味着所有纠错都是被动的LLM 的调用成本照付但产出质量没有随迭代提升。2. 总体设计本地偏差评估器 重试循环2.1 架构在improve_resume()与安全网之间插入一个本地评估器。若评估器发现偏差超过阈值则携带具体反馈重试 LLM 调用最多重试 1 次总共 2 次尝试避免延迟爆炸┌─────────────────────┐ │ retry with feedback │ │ (max 1 retry) │ └────────┬────────────┘ │ ▼ extract_keywords ──→ improve_resume ──→ evaluate_deviation ──→ safety nets ──→ refine ──→ done LLM #1 LLM #2 (local, free) (local) LLM #3 ▲ │ │ fail │ pass └────────────────┘2.2 评估器本地、零 LLM 成本评估器纯本地运行——不调用 LLM。它把 improve 输出与原简历数据逐项对比返回 pass/fail 与具体反馈文本。五类检查全部是字符串/集合比较唯一可能新增的 LLM 成本仅发生在确实检测到偏差并重试这一条路径上。2.3 各组件落位组件文件函数偏差评估器app/services/improver.pyevaluate_deviation()重试循环app/services/improver.pyimprove_resume()内部偏差反馈提示app/prompts/templates.pyDEVIATION_FEEDBACK_SUFFIX富化并行化app/routers/enrichment.pygenerate_enhancements()自动策略选择app/services/improver.pyauto_select_strategy()3. Change 1偏差评估器的五类检查3.1 检查一章节数量保持LLM 不得删减或新增工作经历、教育、项目条目。这是文档点名的最常见失败模式输出变长时LLM 会静默丢掉最早的工作经历条目而用户往往到打印 PDF 时才察觉。def _check_section_counts( original: dict[str, Any], improved: dict[str, Any], ) - list[str]: issues [] for key, label in [ (workExperience, work experience), (education, education), (personalProjects, project), ]: orig_count len(original.get(key, [])) imp_count len(improved.get(key, [])) if imp_count orig_count: issues.append( fDropped {orig_count - imp_count} {label} fentries ({orig_count} → {imp_count}). fKeep ALL original entries. ) elif imp_count orig_count: issues.append( fAdded {imp_count - orig_count} new {label} fentries ({orig_count} → {imp_count}). fDo NOT invent new entries. ) return issues注意该检查是双向的不仅拦截删减imp_count orig_count也拦截凭空新增条目imp_count orig_count。3.2 检查二字数比improve 后的简历字数不应超过原稿 1.8 倍。过度润色会让简历一眼看上去就是 AI 生成的。def _check_word_count( original: dict[str, Any], improved: dict[str, Any], max_ratio: float 1.8, ) - list[str]: orig_words _count_description_words(original) imp_words _count_description_words(improved) if orig_words 0 and imp_words orig_words * max_ratio: return [ fDescription word count increased {imp_words / orig_words:.1f}x f({orig_words} → {imp_words}). fKeep descriptions concise — do not over-elaborate. ] return []为什么是 1.8xfull策略合法地扩充要点2x 以上则几乎必然是过度润色。1.8x 为 full 策略留出余量同时能抓住失控膨胀。源码佐证improver.py中已有_count_description_words()统计 workExperience、personalProjects 的 description 数组与 summary 的字数且verify_diff_result()的第 4 项检查同样使用 1.8 倍阈值result_words orig_words * 1.8可见该阈值在 diff 管线中已作为告警标准落地。3.3 检查三伪造技能检测输出中的新技能必须来自 JD 关键词而非 LLM 凭空发明。def _check_fabricated_skills( original: dict[str, Any], improved: dict[str, Any], job_keywords: dict[str, Any], ) - list[str]: orig_skills _extract_skills_set(original) imp_skills _extract_skills_set(improved) jd_skills _extract_jd_skills_set(job_keywords) fabricated imp_skills - orig_skills - jd_skills if fabricated: # Only flag if more than 1 — single-word variants are common if len(fabricated) 1: sample , .join(sorted(fabricated)[:5]) return [ fAdded skills not in original resume or job description: f{sample}. Only use skills from the original resume or JD. ] return []为什么这能抓到 refiner 漏掉的validate_master_alignment()apps/backend/app/services/refiner.py是对照主简历校验——而主简历的技能可能比本次定向输入更多因此伪造技能有可能绕过它。此检查更严格技能既不在原简历、也不在 JD 中才会被标记。为降低误报仅当伪造项多于 1 个时才触发单词的变体很常见如 Python vs Python 3。3.4 检查四公司与职位保持LLM 不得改名公司或职位头衔。def _check_entry_identity( original: dict[str, Any], improved: dict[str, Any], ) - list[str]: issues [] for key, title_field, subtitle_field, label in [ (workExperience, title, company, work experience), (education, degree, institution, education), ]: orig_entries original.get(key, []) imp_entries improved.get(key, []) for i, (orig, imp) in enumerate(zip(orig_entries, imp_entries)): orig_id ( orig.get(subtitle_field, ).strip().lower() ) imp_id ( imp.get(subtitle_field, ).strip().lower() ) if orig_id and imp_id and orig_id ! imp_id: issues.append( fChanged {label}[{i}] {subtitle_field} from f{orig.get(subtitle_field)} to f{imp.get(subtitle_field)}. fNever change company names, institutions, or degrees. ) return issues该检查使用小写化.lower()做不区分大小写的身份比较避免把 Acme Corp → acme corp 这类无害大小写变化误判为改名。3.5 检查五personalInfo 泄漏improve 提示词明确要求 LLM 跳过 personalInfo该字段会从原简历保留。如果它出现在输出中就是一次偏差。def _check_personal_info_leak(improved: dict[str, Any]) - list[str]: pi improved.get(personalInfo) if pi and isinstance(pi, dict) and any(pi.values()): return [ Output includes personalInfo — the improve prompt excludes it. Do NOT include personalInfo in output. ] return []源码佐证apps/backend/app/prompts/templates.py中IMPROVE_SCHEMA_EXAMPLE明确不含 personalInfo三个策略提示词都写了 Do NOT include personalInfo in your output - it will be preserved from the original resumeimprover.py的_check_for_truncation()注释也说明 personalInfo 被有意排除由_preserve_personal_info()从原简历恢复。检查五是这一约定的最后一环。3.6 组合评估器dataclass class DeviationResult: passed: bool issues: list[str] severity: str # none | soft | hard def evaluate_deviation( original: dict[str, Any], improved: dict[str, Any], job_keywords: dict[str, Any], ) - DeviationResult: Local quality gate — zero LLM cost. Returns pass/fail with specific feedback for retry prompt. issues: list[str] [] # Hard failures — always retry issues.extend(_check_section_counts(original, improved)) issues.extend(_check_entry_identity(original, improved)) hard_issues len(issues) # Soft failures — retry only on first attempt issues.extend(_check_word_count(original, improved)) issues.extend(_check_fabricated_skills(original, improved, job_keywords)) issues.extend(_check_personal_info_leak(improved)) if not issues: return DeviationResult(passedTrue, issues[], severitynone) severity hard if hard_issues 0 else soft return DeviationResult(passedFalse, issuesissues, severityseverity)严重级划分是重试策略的核心硬失败章节数量、条目身份→ 总是重试软失败字数、技能、personalInfo→ 仅记录不强制重试refiner 反正会兜底处理伪造技能。4. Change 2improve_resume()中的重试循环在improve_resume()外部包一层最多 2 次的尝试循环第 1 次原始调用 第 2 次携带反馈的重试async def improve_resume( original_resume: str, job_description: str, job_keywords: dict[str, Any], language: str en, prompt_id: str | None None, original_resume_data: dict[str, Any] | None None, ) - dict[str, Any]: # ... existing setup (lines 170-211) stays the same ... max_attempts 2 last_result None for attempt in range(max_attempts): current_prompt prompt if attempt 0 and last_result is not None: # Append deviation feedback to prompt evaluation evaluate_deviation( original_resume_data or {}, last_result, job_keywords, ) feedback \n.join(f- {issue} for issue in evaluation.issues) current_prompt ( prompt f\n\n--- CRITICAL CORRECTIONS (your previous output had these problems) ---\n feedback \n\nFix ALL issues listed above. Do not repeat these mistakes. ) logger.info( Deviation retry (attempt %d/%d): %s, attempt 1, max_attempts, feedback, ) result await complete_json( promptcurrent_prompt, system_promptYou are an expert resume editor. Output only valid JSON., max_tokens8192, ) _check_for_truncation(result) validated ResumeData.model_validate(result) result_dict validated.model_dump() # Evaluate deviation (skip on last attempt — take what we get) if attempt max_attempts - 1 and original_resume_data: evaluation evaluate_deviation( original_resume_data, result_dict, job_keywords, ) if evaluation.passed: return result_dict # Only retry on hard failures (dropped entries, renamed companies) # Soft failures (word count, skills) proceed with warning if evaluation.severity soft: logger.warning( Soft deviation detected (not retrying): %s, evaluation.issues, ) return result_dict # Hard failure — retry logger.warning( Hard deviation detected (retrying): %s, evaluation.issues, ) last_result result_dict continue return result_dict # Should not reach here, but safety fallback return result_dict4.1 关键设计决策最多 1 次重试共 2 次尝试重试是叠加延迟的。一次 improve 调用耗时 8–12s重试一次再加 8–12s而整个流程本身已有 240s 超时预算加太多重试会吃掉安全余量。硬/软严重级分流丢失条目与改名公司总是重试字数与技能问题只产生警告——伪造技能反正会被 refiner 处理。反馈进提示词重试把具体问题追加到原提示词后面LLM 能精确看到自己哪里错了。这比泛泛的 try again 有效得多。评估器在安全网之前运行resumes.py的安全网_preserve_personal_info、_restore_original_dates等是事后打补丁的。评估器必须作用于 LLM 的原始输出打补丁之前因此它住在improve_resume()内部而不是 router 层。评估零 LLM 成本5 项检查全是本地字符串/集合比较唯一的额外 LLM 成本是检测到偏差时的重试调用本身。4.2 与传输层重试的协作关系注意区分两个层次llm.py:complete_json()的重试针对结构故障JSON 解析失败、截断、空响应本方案的重试针对内容偏差结构合法但内容失实。两者互补传输层保证格式对评估器保证内容对。设计文档也明确llm.py不在改动范围内。5. Change 3偏差反馈提示后缀不需要新增提示词模板。反馈由评估器的 issues 列表动态构造作为后缀追加到既有提示词上保持提示词系统精简、无新增模板需要维护。后缀格式--- CRITICAL CORRECTIONS (your previous output had these problems) --- - Dropped 1 work experience entry (4 → 3). Keep ALL original entries. - Changed workExperience[0] company from Acme Corp to ACME Corporation. Never change company names. Fix ALL issues listed above. Do not repeat these mistakes.每个 issue 一行- {issue}用\n连接落点即重试循环中current_prompt prompt 后缀 收尾指令的拼接逻辑。6. Change 4generate_enhancements()并行化app/routers/enrichment.py的富化端点原来对每个条目串行调用complete_jsonfor item_id, answers in answers_by_item.items(): # ... build prompt ... result await complete_json(prompt) # blocks on each item enhancements.append(...)改为asyncio.gather并行async def _enhance_single_item(item_id, item, answers, questions_by_id): Generate enhanced descriptions for a single item. # ... build prompt (existing code) ... result await complete_json(prompt) additional_bullets result.get(additional_bullets, []) if not additional_bullets: additional_bullets result.get(enhanced_description, []) if not isinstance(additional_bullets, list): additional_bullets [] additional_bullets [str(b) for b in additional_bullets if b] return EnhancedDescription( item_iditem_id, item_typeitem.get(item_type, experience), titleitem.get(title, ), original_descriptionitem.get(current_description, []), enhanced_descriptionadditional_bullets, ) # In generate_enhancements(): tasks [ _enhance_single_item(item_id, item_details.get(item_id, {}), answers, questions_by_id) for item_id, answers in answers_by_item.items() if item_details.get(item_id) ] results await asyncio.gather(*tasks, return_exceptionsTrue) enhancements [] for result in results: if isinstance(result, Exception): logger.warning(fFailed to enhance item: {result}) else: enhancements.append(result)影响若用户有 4 个条目待富化延迟从约 40s4 次串行调用降到约 10s1 批并行。从当前源码看apps/backend/app/routers/enrichment.py的generate_enhancements()仍是逐条for循环调用complete_json说明并行化仍是待落地的优化提案不过regenerate_items端点已经使用了asyncio.gather(*tasks, return_exceptionsTrue)模式每个条目一个协程、异常隔离、失败只记录不中断可作为并行化落地的同仓库先例。设计文档同时建议用return_exceptionsTrue保证单条目失败不拖垮整批。7. Change 5自动策略选择新增auto_select_strategy()按关键词匹配率在nudge/keywords/full之间选择供prompt_idauto时使用def auto_select_strategy( original_resume_data: dict[str, Any], job_keywords: dict[str, Any], ) - str: Select tailoring strategy based on keyword match percentage. Uses the same keyword matching logic as the refiner to determine how much work the LLM needs to do. from app.services.refiner import calculate_keyword_match match_pct calculate_keyword_match(original_resume_data, job_keywords) if match_pct 70: return nudge # already close — light rephrasing elif match_pct 35: return keywords # relevant but missing key terms else: return full # needs significant restructuring集成点在improve_resume()selected_prompt_id prompt_id or DEFAULT_IMPROVE_PROMPT_ID if selected_prompt_id auto and original_resume_data: selected_prompt_id auto_select_strategy(original_resume_data, job_keywords) logger.info(Auto-selected strategy: %s, selected_prompt_id)这需要在app/prompts/templates.py的IMPROVE_PROMPT_OPTIONS列表中加入auto前端才能提供该选项。源码佐证templates.py中IMPROVE_PROMPT_OPTIONS目前只含 nudge / keywords / full 三项DEFAULT_IMPROVE_PROMPT_ID keywordscalculate_keyword_match()已在app/services/refiner.py中实现test_refiner.py中TestCalculateKeywordMatch验证了 0–100 的百分比语义与词边界匹配如 Go 不应匹配 Google。策略阈值语义≥70% 说明简历与 JD 已经很接近轻改写即可35%–70% 说明相关但缺关键术语35% 说明需要整体重构。8. 变更总览、不变量与延迟影响8.1 全部变更#内容位置类型LLM 成本1evaluate_deviation()— 5 项本地检查app/services/improver.py新函数02improve_resume()中的重试循环app/services/improver.py修改既有仅硬失败时 1 次调用3偏差反馈后缀动态构造improve_resume()内无新模板04generate_enhancements()并行化app/routers/enrichment.py修改既有05auto_select_strategy()app/services/improver.py新函数08.2 明确不动的部分llm.py—— 传输层重试与 JSON 质量检查保持不变refiner.py—— 对齐校验、AI 短语移除、关键词注入保持不变resumes.py安全网 ——_preserve_personal_info、_restore_original_dates等保持不变不新增依赖不用 LangChain不引入工作流引擎不新增提示词模板 —— 反馈动态构造。8.3 延迟影响场景现在改动后快乐路径无偏差~12s~12s评估器增加 1ms硬偏差丢条目~12s坏输出~24s重试产出正确输出富化4 个条目~40s~10s自动策略N/A1ms本地关键词匹配9. 实施顺序evaluate_deviation() 重试循环—— 影响最大先拦最严重的失败富化并行化—— 直接了当立刻拿到延迟收益自动策略选择—— 小改动但需要前端支持 auto 选项。10. 设计演进从全量输出偏差检测到 diff-based 管线本文依据的设计文档状态标注为Superseded——其方案已被 diff-based improvement design 取代。理解这段演进对读者非常重要因为它解释了同一批偏差问题在现仓库中如何被结构性地解决核心思路转变不再让 LLM 输出整份简历全量输出 每个字段都是幻觉机会而是只输出想改什么的定向 diff。原简历结构程序化保留偏差问题从源头消除丢条目/改名公司/日期截断/personalInfo 泄漏/customSections 伪造 →在 applier 中被按构造消除apply_diffs()的路径白名单_ALLOWED_PATH_PATTERNS与黑名单_BLOCKED_PATH_PREFIXES、_BLOCKED_FIELD_NAMES直接禁止触碰 identity 字段伪造技能/发明指标 →显式 diff 本地校验器捕获。现仓库apps/backend/app/services/improver.py的实际管线是generate_resume_diffs() ──→ apply_diffs() ──→ verify_diff_result() ──→ refine_resume() LLM 本地 本地 LLM其中verify_diff_result()就是本设计文档思想在现管线中的落点。它实现了与evaluate_deviation()高度重合的本地质量检查apps/backend/app/services/improver.py无变更应用时告警No changes were applied章节数量保持workExperience / education / personalProjects 计数一致身份字段不变company、title、institution、degree 逐项比对字数比不超过 1.8x发明指标检测\d%、\dx、$\d正则新增数字若不在原文中则告警。测试佐证见 test_verify_diffs.py覆盖丢弃工作经历/教育/项目条目告警改名公司/头衔/院校告警字数翻倍告警新增 40% / $5M 指标告警正常增长不告警等用例第 5 项发明指标检测正是本设计文档检查三/检查四之外额外补上的新维度。在编排侧app/routers/resumes.py的_improve_preview_flow()现在依次调用generate_resume_diffs → apply_diffs → verify_diff_result随后仍执行安全网_preserve_personal_info、_restore_original_dates、restore_dates_from_markdown、_preserve_original_skills、_protect_custom_sections与多轮 refinement——即文档中6 道安全网 refine在 diff 时代作为纵深防御保留了下来。值得注意设计文档中 Change 4富化并行化与 Change 5auto 策略并不属于 diff 重构的范围两者至今仍是未落地的优化提案generate_enhancements()仍串行、IMPROVE_PROMPT_OPTIONS仍无 auto任何希望贡献的开发者都可以从这里入手。11. 总结这份设计文档回答了一个贯穿 Resume-Matcher 优化管线始终的问题如何让 LLM 输出结构正确且内容忠实。它给出的方案——本地零成本评估器 硬/软严重级重试 动态反馈后缀——虽然在后续演进中被 diff-based 方案以从源头消除偏差的方式超越但其检查维度章节数量、字数比、伪造技能、身份字段、personalInfo 泄漏全部沉淀为现仓库verify_diff_result()的本地校验逻辑并被测试固化为回归保障。而富化并行化与自动策略选择两个提案至今仍有明确的落地空间。对想深入源码的读者建议按以下路径阅读管线编排apps/backend/app/routers/resumes.py的_improve_preview_flow()本地校验apps/backend/app/services/improver.py的apply_diffs()、verify_diff_result()与generate_resume_diffs()传输层重试与温度策略apps/backend/app/llm.py的complete_json()、_get_retry_temperature()、_supports_temperature()对齐校验与关键词匹配apps/backend/app/services/refiner.py的validate_master_alignment()、calculate_keyword_match()测试回归apps/backend/tests/unit/test_verify_diffs.py、apps/backend/tests/unit/test_refiner.py后续设计docs/superpowers/specs/2026-03-23-diff-based-improvement-design.md【免费下载链接】Resume-MatcherThe #1 AI Harness for Building Resumes, PDFs, Cover Letters more, locally with 100 LLMs support.项目地址: https://gitcode.com/GitHub_Trending/re/Resume-Matcher创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考