Selig
4c966a3ad2
6 custom skills (assign-task, dispatch-webhook, daily-briefing, task-capture, qmd-brain, tts-voice) with technical documentation. Compatible with Claude Code, OpenClaw, Codex CLI, and OpenCode.
153 lines
4.2 KiB
Markdown
Executable File
153 lines
4.2 KiB
Markdown
Executable File
# OpenClaw 文檔抓取專案總結
|
||
|
||
**日期**: 2024-02-09
|
||
**專案**: OpenClaw AI Agent 平台文檔本地化
|
||
|
||
---
|
||
|
||
## 專案目標
|
||
|
||
將 [OpenClaw 官方文檔](https://docs.openclaw.ai/) 完整抓取並按章節組織存入本地，建立離線可用的文檔資料庫，為後續製作 Claude Skill 做準備。
|
||
|
||
---
|
||
|
||
## 完成成果
|
||
|
||
### 文檔抓取統計
|
||
|
||
| 章節 | 檔案數 | 說明 |
|
||
|------|--------|------|
|
||
| get-started | 11 | 快速開始、安裝設定 |
|
||
| channels | 18 | WhatsApp、Telegram、Discord 等通訊整合 |
|
||
| agents | 5 | Agent 運行時、多代理路由 |
|
||
| tools | 18 | 瀏覽器控制、Skills、CLI 工具 |
|
||
| models | 13 | Anthropic、OpenAI、OpenRouter 等模型供應商 |
|
||
| help | 3 | FAQ、故障排除 |
|
||
| infrastructure/gateway | 28 | Gateway 架構、設定、安全性 |
|
||
| infrastructure/install | 9 | Docker、Nix、Ansible 安裝方式 |
|
||
| infrastructure/nodes | 7 | 音訊、相機、語音喚醒 |
|
||
| infrastructure/web | 4 | Control UI、Dashboard、WebChat |
|
||
| infrastructure/automation | 6 | Cron、Webhook、Gmail PubSub |
|
||
| infrastructure/platforms | 29 | macOS、iOS、Android、Windows、Linux |
|
||
| infrastructure/其他 | 18 | Hooks、Plugins、Security 等 |
|
||
| reference/cli | 36 | 完整 CLI 命令參考 |
|
||
| reference/concepts | 27 | 核心概念：Session、Memory、Streaming 等 |
|
||
| reference/其他 | 5 | Credits、RPC、測試 |
|
||
|
||
**總計: 237 個 Markdown 檔案，825 KiB**
|
||
|
||
---
|
||
|
||
## 技術突破：fetch_and_save 工具
|
||
|
||
### 問題發現
|
||
|
||
使用 WebFetch 抓取網頁時，內容需經過 Claude API 傳輸，每頁消耗約 2000 tokens。抓取 200+ 頁文檔將消耗大量 token 配額。
|
||
|
||
### 解決方案
|
||
|
||
為 Skill Seeker MCP 開發了 `fetch_and_save` 工具：
|
||
|
||
```python
|
||
# 直接用 Python httpx 下載並存檔，不經過 Claude API
|
||
fetch_and_save(
|
||
url="https://docs.example.com/guide.md",
|
||
output="docs/guide.md"
|
||
)
|
||
|
||
# 批次模式
|
||
fetch_and_save(urls=[
|
||
{"url": "https://...", "output": "..."},
|
||
{"url": "https://...", "output": "..."}
|
||
])
|
||
```
|
||
|
||
### 效益比較
|
||
|
||
| 方式 | Token 消耗/頁 | 200 頁總消耗 |
|
||
|------|--------------|-------------|
|
||
| WebFetch + Write | ~2,000 | 400,000 |
|
||
| **fetch_and_save** | **~50** | **10,000** |
|
||
|
||
**節省 97.5% 的 token 成本！**
|
||
|
||
### 修改的檔案
|
||
|
||
```
|
||
skill_seekers/mcp/tools/scraping_tools.py ← 新增 fetch_and_save_tool
|
||
skill_seekers/mcp/tools/__init__.py ← 導出新工具
|
||
skill_seekers/mcp/server_fastmcp.py ← 註冊 MCP 工具
|
||
```
|
||
|
||
### 可攜性解決方案
|
||
|
||
建立了 patch 腳本，可在任何電腦上快速部署：
|
||
|
||
```
|
||
patches/
|
||
├── add_fetch_and_save.py ← 自動 patch 腳本
|
||
└── README.md ← 使用說明
|
||
```
|
||
|
||
使用方式：
|
||
```bash
|
||
pip install skill-seekers
|
||
python add_fetch_and_save.py
|
||
# 重啟 Claude Code
|
||
```
|
||
|
||
---
|
||
|
||
## 專案結構
|
||
|
||
```
|
||
D:\openclaw\skills\openclaw\
|
||
├── SKILL.md ← 原有的 OpenClaw Skill 文檔
|
||
├── Summarize-0209.md ← 本次工作總結
|
||
├── patches/
|
||
│ ├── add_fetch_and_save.py ← Skill Seeker patch 腳本
|
||
│ └── README.md
|
||
└── docs/ ← 抓取的完整文檔 (237 檔案)
|
||
├── get-started/
|
||
├── channels/
|
||
├── agents/
|
||
├── tools/
|
||
├── models/
|
||
├── help/
|
||
├── infrastructure/
|
||
│ ├── gateway/
|
||
│ ├── install/
|
||
│ ├── nodes/
|
||
│ ├── web/
|
||
│ ├── automation/
|
||
│ ├── platforms/
|
||
│ │ └── mac/
|
||
│ ├── hooks/
|
||
│ ├── plugins/
|
||
│ └── security/
|
||
└── reference/
|
||
├── cli/
|
||
└── concepts/
|
||
```
|
||
|
||
---
|
||
|
||
## 下一步
|
||
|
||
1. 利用抓取的文檔建立完整的 OpenClaw Claude Skill
|
||
2. 將 `fetch_and_save` 工具提交 PR 到 Skill Seeker 上游專案
|
||
3. 持續更新文檔（重新執行抓取腳本即可）
|
||
|
||
---
|
||
|
||
## 經驗總結
|
||
|
||
1. **批次抓取文檔時，應避免讓內容經過 LLM API** — 直接用 Python 處理更高效
|
||
2. **平行代理可加速抓取**，但需注意 API 配額限制
|
||
3. **建立 patch 腳本**確保工具可在其他環境重現
|
||
4. **llms.txt 文件**可快速獲取網站完整頁面清單
|
||
|
||
---
|
||
|
||
*Generated with Claude Code (Opus 4.5) on 2024-02-09*
|