{
  "schemaVersion": "1.0",
  "item": {
    "slug": "memory-dedup",
    "name": "Memory Deduplication",
    "source": "tencent",
    "type": "skill",
    "category": "AI 智能",
    "sourceUrl": "https://clawhub.ai/weidadong2359/memory-dedup",
    "canonicalUrl": "https://clawhub.ai/weidadong2359/memory-dedup",
    "targetPlatform": "OpenClaw"
  },
  "install": {
    "downloadMode": "redirect",
    "downloadUrl": "/downloads/memory-dedup",
    "sourceDownloadUrl": "https://wry-manatee-359.convex.site/api/v1/download?slug=memory-dedup",
    "sourcePlatform": "tencent",
    "targetPlatform": "OpenClaw",
    "installMethod": "Manual import",
    "extraction": "Extract archive",
    "prerequisites": [
      "OpenClaw"
    ],
    "packageFormat": "ZIP package",
    "includedAssets": [
      "SKILL.md",
      "dedup.mjs",
      "package.json"
    ],
    "primaryDoc": "SKILL.md",
    "quickSetup": [
      "Download the package from Yavira.",
      "Extract the archive and review SKILL.md first.",
      "Import or place the package into your OpenClaw setup."
    ],
    "agentAssist": {
      "summary": "Hand the extracted package to your coding agent with a concrete install brief instead of figuring it out manually.",
      "steps": [
        "Download the package from Yavira.",
        "Extract it into a folder your agent can access.",
        "Paste one of the prompts below and point your agent at the extracted folder."
      ],
      "prompts": [
        {
          "label": "New install",
          "body": "I downloaded a skill package from Yavira. Read SKILL.md from the extracted folder and install it by following the included instructions. Tell me what you changed and call out any manual steps you could not complete."
        },
        {
          "label": "Upgrade existing",
          "body": "I downloaded an updated skill package from Yavira. Read SKILL.md from the extracted folder, compare it with my current installation, and upgrade it while preserving any custom configuration unless the package docs explicitly say otherwise. Summarize what changed and any follow-up checks I should run."
        }
      ]
    },
    "sourceHealth": {
      "source": "tencent",
      "status": "healthy",
      "reason": "direct_download_ok",
      "recommendedAction": "download",
      "checkedAt": "2026-04-30T16:55:25.780Z",
      "expiresAt": "2026-05-07T16:55:25.780Z",
      "httpStatus": 200,
      "finalUrl": "https://wry-manatee-359.convex.site/api/v1/download?slug=network",
      "contentType": "application/zip",
      "probeMethod": "head",
      "details": {
        "probeUrl": "https://wry-manatee-359.convex.site/api/v1/download?slug=network",
        "contentDisposition": "attachment; filename=\"network-1.0.0.zip\"",
        "redirectLocation": null,
        "bodySnippet": null
      },
      "scope": "source",
      "summary": "Source download looks usable.",
      "detail": "Yavira can redirect you to the upstream package for this source.",
      "primaryActionLabel": "Download for OpenClaw",
      "primaryActionHref": "/downloads/memory-dedup"
    },
    "validation": {
      "installChecklist": [
        "Use the Yavira download entry.",
        "Review SKILL.md after the package is downloaded.",
        "Confirm the extracted package contains the expected setup assets."
      ],
      "postInstallChecks": [
        "Confirm the extracted package includes the expected docs or setup files.",
        "Validate the skill or prompts are available in your target agent workspace.",
        "Capture any manual follow-up steps the agent could not complete."
      ]
    },
    "downloadPageUrl": "https://openagent3.xyz/downloads/memory-dedup",
    "agentPageUrl": "https://openagent3.xyz/skills/memory-dedup/agent",
    "manifestUrl": "https://openagent3.xyz/skills/memory-dedup/agent.json",
    "briefUrl": "https://openagent3.xyz/skills/memory-dedup/agent.md"
  },
  "agentAssist": {
    "summary": "Hand the extracted package to your coding agent with a concrete install brief instead of figuring it out manually.",
    "steps": [
      "Download the package from Yavira.",
      "Extract it into a folder your agent can access.",
      "Paste one of the prompts below and point your agent at the extracted folder."
    ],
    "prompts": [
      {
        "label": "New install",
        "body": "I downloaded a skill package from Yavira. Read SKILL.md from the extracted folder and install it by following the included instructions. Tell me what you changed and call out any manual steps you could not complete."
      },
      {
        "label": "Upgrade existing",
        "body": "I downloaded an updated skill package from Yavira. Read SKILL.md from the extracted folder, compare it with my current installation, and upgrade it while preserving any custom configuration unless the package docs explicitly say otherwise. Summarize what changed and any follow-up checks I should run."
      }
    ]
  },
  "documentation": {
    "source": "clawhub",
    "primaryDoc": "SKILL.md",
    "sections": [
      {
        "title": "Memory Deduplication — 记忆去重与合并",
        "body": "保持 MEMORY.md 清洁，避免信息冗余"
      },
      {
        "title": "问题",
        "body": "随着时间推移，MEMORY.md 会出现：\n\n重复信息 — 同一事件多次记录\n过时信息 — 已完成的任务仍标记为进行中\n冗余描述 — 同一项目多处描述\n碎片化 — 相关信息散落各处"
      },
      {
        "title": "1. 自动去重",
        "body": "识别并合并相似内容：\n\n原始:\n- AgentAwaken 网站开发中\n- AgentAwaken 项目进行中\n- AgentAwaken 待部署\n\n合并后:\n- AgentAwaken 网站: 开发中，待部署"
      },
      {
        "title": "2. 过时信息清理",
        "body": "原始:\n- [P0] NeuroBoost v5.0 发布待重试\n\n更新后:\n- [P0] NeuroBoost v5.0 ✅ 已发布 (2026-02-26)"
      },
      {
        "title": "3. 信息聚合",
        "body": "原始:\n### AgentAwaken\n- 代码: /root/.openclaw/workspace/agentawaken\n### AgentAwaken 域名\n- agentawaken.xyz 待绑定\n### AgentAwaken 部署\n- 需要 Vercel\n\n合并后:\n### [P0] AgentAwaken 网站\n- 代码: /root/.openclaw/workspace/agentawaken\n- 域名: agentawaken.xyz (待绑定)\n- 部署: Vercel (待配置)"
      },
      {
        "title": "相似度计算",
        "body": "function similarity(text1, text2) {\n  // Jaccard 相似度\n  const words1 = new Set(text1.toLowerCase().split(/\\s+/));\n  const words2 = new Set(text2.toLowerCase().split(/\\s+/));\n  const intersection = new Set([...words1].filter(x => words2.has(x)));\n  const union = new Set([...words1, ...words2]);\n  return intersection.size / union.size;\n}"
      },
      {
        "title": "去重规则",
        "body": "相似度 >0.8 — 完全重复，删除\n相似度 0.5-0.8 — 部分重复，合并\n相似度 <0.5 — 不同内容，保留"
      },
      {
        "title": "合并策略",
        "body": "保留最新时间戳\n合并所有唯一信息\n保留最高优先级标记"
      },
      {
        "title": "使用",
        "body": "# 运行去重\nnode skills/memory-dedup/dedup.mjs\n\n# 预览（不修改文件）\nnode skills/memory-dedup/dedup.mjs --dry-run\n\n# 备份后去重\nnode skills/memory-dedup/dedup.mjs --backup"
      },
      {
        "title": "输出示例",
        "body": "=== Memory Deduplication Report ===\n\n📊 统计:\n- 原始条目: 87\n- 重复条目: 12\n- 合并条目: 5\n- 删除条目: 7\n- 最终条目: 68\n\n🔍 发现的重复:\n1. \"AgentAwaken 网站开发\" (3 次)\n   → 合并为 1 条\n2. \"NeuroBoost v5.0 发布\" (2 次)\n   → 保留最新版本\n\n✅ MEMORY.md 已优化\n💾 备份保存到: memory/MEMORY-backup-2026-03-01.md"
      },
      {
        "title": "定期执行",
        "body": "# 每周日凌晨 2 点自动去重\nopenclaw cron add --name \"memory-dedup-weekly\" \\\n  --cron \"0 2 * * 0\" --tz \"Asia/Shanghai\" \\\n  --session isolated --agent main \\\n  --message \"运行记忆去重，清理 MEMORY.md 冗余信息\""
      },
      {
        "title": "安全措施",
        "body": "自动备份 — 去重前备份原文件\n人工审核 — 生成 diff 供审核\n可回滚 — 保留最近 10 次备份\n白名单 — 某些关键信息不去重"
      },
      {
        "title": "效果",
        "body": "文件大小减少 30-50%\n检索速度提升 2-3 倍\n信息密度提升 40%\n维护成本降低 60%"
      }
    ],
    "body": "Memory Deduplication — 记忆去重与合并\n\n保持 MEMORY.md 清洁，避免信息冗余\n\n问题\n\n随着时间推移，MEMORY.md 会出现：\n\n重复信息 — 同一事件多次记录\n过时信息 — 已完成的任务仍标记为进行中\n冗余描述 — 同一项目多处描述\n碎片化 — 相关信息散落各处\n解决方案\n1. 自动去重\n\n识别并合并相似内容：\n\n原始:\n- AgentAwaken 网站开发中\n- AgentAwaken 项目进行中\n- AgentAwaken 待部署\n\n合并后:\n- AgentAwaken 网站: 开发中，待部署\n\n2. 过时信息清理\n原始:\n- [P0] NeuroBoost v5.0 发布待重试\n\n更新后:\n- [P0] NeuroBoost v5.0 ✅ 已发布 (2026-02-26)\n\n3. 信息聚合\n原始:\n### AgentAwaken\n- 代码: /root/.openclaw/workspace/agentawaken\n### AgentAwaken 域名\n- agentawaken.xyz 待绑定\n### AgentAwaken 部署\n- 需要 Vercel\n\n合并后:\n### [P0] AgentAwaken 网站\n- 代码: /root/.openclaw/workspace/agentawaken\n- 域名: agentawaken.xyz (待绑定)\n- 部署: Vercel (待配置)\n\n实现\n相似度计算\nfunction similarity(text1, text2) {\n  // Jaccard 相似度\n  const words1 = new Set(text1.toLowerCase().split(/\\s+/));\n  const words2 = new Set(text2.toLowerCase().split(/\\s+/));\n  const intersection = new Set([...words1].filter(x => words2.has(x)));\n  const union = new Set([...words1, ...words2]);\n  return intersection.size / union.size;\n}\n\n去重规则\n相似度 >0.8 — 完全重复，删除\n相似度 0.5-0.8 — 部分重复，合并\n相似度 <0.5 — 不同内容，保留\n合并策略\n保留最新时间戳\n合并所有唯一信息\n保留最高优先级标记\n使用\n# 运行去重\nnode skills/memory-dedup/dedup.mjs\n\n# 预览（不修改文件）\nnode skills/memory-dedup/dedup.mjs --dry-run\n\n# 备份后去重\nnode skills/memory-dedup/dedup.mjs --backup\n\n输出示例\n=== Memory Deduplication Report ===\n\n📊 统计:\n- 原始条目: 87\n- 重复条目: 12\n- 合并条目: 5\n- 删除条目: 7\n- 最终条目: 68\n\n🔍 发现的重复:\n1. \"AgentAwaken 网站开发\" (3 次)\n   → 合并为 1 条\n2. \"NeuroBoost v5.0 发布\" (2 次)\n   → 保留最新版本\n\n✅ MEMORY.md 已优化\n💾 备份保存到: memory/MEMORY-backup-2026-03-01.md\n\n定期执行\n# 每周日凌晨 2 点自动去重\nopenclaw cron add --name \"memory-dedup-weekly\" \\\n  --cron \"0 2 * * 0\" --tz \"Asia/Shanghai\" \\\n  --session isolated --agent main \\\n  --message \"运行记忆去重，清理 MEMORY.md 冗余信息\"\n\n安全措施\n自动备份 — 去重前备份原文件\n人工审核 — 生成 diff 供审核\n可回滚 — 保留最近 10 次备份\n白名单 — 某些关键信息不去重\n效果\n文件大小减少 30-50%\n检索速度提升 2-3 倍\n信息密度提升 40%\n维护成本降低 60%"
  },
  "trust": {
    "sourceLabel": "tencent",
    "provenanceUrl": "https://clawhub.ai/weidadong2359/memory-dedup",
    "publisherUrl": "https://clawhub.ai/weidadong2359/memory-dedup",
    "owner": "weidadong2359",
    "version": "1.0.0",
    "license": null,
    "verificationStatus": "Indexed source record"
  },
  "links": {
    "detailUrl": "https://openagent3.xyz/skills/memory-dedup",
    "downloadUrl": "https://openagent3.xyz/downloads/memory-dedup",
    "agentUrl": "https://openagent3.xyz/skills/memory-dedup/agent",
    "manifestUrl": "https://openagent3.xyz/skills/memory-dedup/agent.json",
    "briefUrl": "https://openagent3.xyz/skills/memory-dedup/agent.md"
  }
}