User directive: daily not weekly; clear responsibilities per layer with no
overlap; functional criteria (does the function WORK) not process liveness;
problems get FIXED via LLM with file-and-report discipline (act first,
report after); plus a meta-layer watching the watchers; deeply integrated
into F健康.
Architecture (responsibility matrix in dev-spec.md):
- L0 agents_health_check (5min): port/HTTP/DB liveness + auto_heal executor
- L1 functional_health_check (15min trading): per-module FUNCTIONAL
criteria — output freshness/validity per REGISTRY (live_prices/market_
snapshots/mtf_cache/macro_context/bot/LLM/cron engine), not process alive
- L2 system_hygiene_audit (daily 08:20, was weekly): divergence/hardlink/
zombie/orphan/dead-cron/db-freshness
- L3 self_repair (30min): reads L1/L2 failures -> LLM diagnoses -> executes
WHITELISTED repair actions directly (rerun_script/restart_service/
sync_links/switch_llm_key/none) -> repair_log.jsonl + XMPP report.
Max 2 repairs/module/day anti-loop. LLM unavailable -> rule fallback.
- L4 meta_watchdog (hourly): checks L0-L3 output freshness + L3 cron
registration + XMPP bridge; direct XMPP alert as last resort
Retired (overlap): Cron监护-高频 (cron_watchdog -> L3), 全局cron健康监控
(cron_health_monitor -> L1).
Dashboard: mofin_health.py now emits self_check section (functional/meta/
hygiene/recent_repairs); mofin_health.html new '🩺 自检体系' tab rendering
L4 layers, L1 module checks, L2 issues, L3 repair history.
E2E verified: stopped xmpp bot -> L1 flagged fail -> systemd recovered ->
L3 LLM correctly diagnosed 'none needed' and logged; rerun_script whitelist
path executes real scripts successfully; meta_watchdog all-green after fix.
65 lines
3.4 KiB
Python
65 lines
3.4 KiB
Python
import json, urllib.request
|
||
|
||
msg = """[笑笑] 系统大扫除 + 防复发机制 — 全部完成
|
||
|
||
■ 清理执行结果
|
||
|
||
【小果生态 — 全灭】
|
||
- xmpp_xiaoguo_bot(root, 8天2.5GB) 已杀,3个systemd服务已删除
|
||
- 4个 cron job 已删(quick-scan/tunnel-watchdog/情感分析/独立扫描)
|
||
- 7个脚本 + 数据文件 + profile目录 全部归档 archive/xiaoguo-retired-20260720/
|
||
|
||
【第三库 — 已合并】
|
||
- sector_snapshots +4978行、market_snapshots +187行、todos +2 并入主库
|
||
- 第三库及其数据目录已移 trashbox
|
||
|
||
【文件源统一 — 31个分叉全处理】
|
||
- 库文件(strategy_lifecycle等5个):统一到 MoFin root(cron实际import的版本)
|
||
- cron脚本(13个):deploy为唯一源,旧副本全归档
|
||
- 新增 sync_profile_scripts.sh 防硬链接断裂
|
||
- crontab 三处改指 deploy 路径(cron_to_xmpp/market_watch/market_screener),已验证可跑
|
||
|
||
【其余清理】
|
||
- 21个遗留JSON + 3个废弃小库(market.db/market_data.db/stock_analysis.db) 归档
|
||
- 8个禁用cron任务删除(pa 52 jobs / default 17 jobs,禁用项清零)
|
||
- 9个旧日志 + 旧bot/bak文件归档
|
||
- projects/ 两个旧克隆(1.4G+2.2M) 归档
|
||
|
||
■ 问题是怎么产生的(根因)
|
||
|
||
1. 没有单一事实源规则——同一文件可以"合法"住在4个地方,悄悄分叉
|
||
2. 相对路径解析——mofin_db.py 用 __file__.parent 找数据目录,每个硬链接副本解析出不同的库(三库事件根源)
|
||
3. "备份留生产目录"的习惯——.bak/迁移残留留在数据目录,被监控扫描后变成假警报
|
||
4. 迁移只做一半——DB建了表,旧JSON的读写着还留着(price_events 就是这么分裂的)
|
||
5. 模块死了没人收尸——小果宣布"已废弃",但bot/gateway/tunnel/cron照跑
|
||
6. 监控查"在不在"不查"活不活"——拿遗留文件mtime当管道健康,报出"停滞14天"假警报
|
||
7. 压根没有"系统卫生"这个检查类别——分叉/僵尸/孤儿文件没人看
|
||
|
||
■ 防复发机制(已落地)
|
||
|
||
【规范层】dev-spec.md 升级 v2.0:五条红线 → 十条红线
|
||
#6 单一事实源(只许硬链接,禁止独立副本)
|
||
#7 数据路径必须绝对(禁止__file__相对解析数据路径)
|
||
#8 备份/遗留物禁止留生产数据目录
|
||
#9 死模块收尸六步清单(杀进程/停服务/删cron/归档脚本/归档数据/摘监控)
|
||
#10 监控查活不查在(DB表新鲜度才算健康证据)
|
||
+ 文件位置宪法:每类内容的唯一权威位置表
|
||
|
||
【监控层】新增 system_hygiene_audit.py
|
||
- 每周一 07:30 自动跑:分叉副本/断裂硬链接/僵尸进程/孤儿文件/死cron/DB新鲜度 六项检查
|
||
- 结果写 hygiene_report.json,有问题自动推 XMPP
|
||
- 已注册进 cron(pa jobs.json)
|
||
- specs/hygiene.json 模块 spec 已就位
|
||
- 首轮实测:发现5个真问题(1个漏网分叉+2个我清理时误删的default脚本+2个合法容器进程误报),已全部修复,复检六项全绿
|
||
|
||
这套机制保证:下次再出现"这一行那也可以"的冗余,周一早上 07:30 你就会收到 XMPP 点名。
|
||
|
||
代码已提交合并(246: c545a820)。"""
|
||
|
||
payload = json.dumps({"to": "hmo@yoin.fun", "body": msg, "type": "chat"}).encode()
|
||
req = urllib.request.Request("http://127.0.0.1:5805/", data=payload,
|
||
headers={"Content-Type": "application/json"})
|
||
try:
|
||
print("XMPP:", urllib.request.urlopen(req, timeout=10).read().decode()[:80])
|
||
except Exception as e:
|
||
print("XMPP fail:", e) |