From 2428e975eb5146ab9264d0e93401f07d63ec5810 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E6=9C=A8=E7=82=8E?= <635735027@qq.com> Date: Sun, 26 Apr 2026 15:02:30 +0800 Subject: [PATCH] docs: add normalized result reader design --- ...esult-and-dashboard-local-reader-design.md | 1188 +++++++++++++++++ 1 file changed, 1188 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-26-skill-normalized-result-and-dashboard-local-reader-design.md diff --git a/docs/superpowers/specs/2026-04-26-skill-normalized-result-and-dashboard-local-reader-design.md b/docs/superpowers/specs/2026-04-26-skill-normalized-result-and-dashboard-local-reader-design.md new file mode 100644 index 0000000..97f988b --- /dev/null +++ b/docs/superpowers/specs/2026-04-26-skill-normalized-result-and-dashboard-local-reader-design.md @@ -0,0 +1,1188 @@ +# Skill Normalized Result And Dashboard Local Reader Design + +Date: 2026-04-26 + +Status: Proposed design for implementation + +Related baseline designs: + +- `docs/superpowers/specs/2026-04-22-scheduled-monitoring-action-end-to-end-architecture-design.md` +- `docs/superpowers/specs/2026-04-23-archive-workorder-grid-push-monitor-design.md` +- `docs/superpowers/specs/2026-04-24-available-balance-below-zero-monitor-design.md` +- `docs/superpowers/specs/2026-04-26-scheduled-monitoring-generated-scene-adaptation-hardening-design.md` + +Related skill/result sources: + +- `D:/desk/sgclaw/sgclaw/results/archive-workorder-grid-push-monitor.run-record.json` +- `D:/desk/sgclaw/sgclaw/results/available-balance-below-zero-monitor.run-record.json` +- `D:/desk/sgclaw/sgclaw/results/command-center-fee-control-monitor.run-record.json` +- `D:/desk/sgclaw/sgclaw/results/sgcc-todo-crawler.run-record.json` + +Target dashboard project: + +- `D:/data/ideaSpace/rust/sgClaw/digital-employee` + +## Plain-Language Goal + +Turn raw scheduled-skill run results into a stable, dashboard-friendly result contract. + +The system must support: + +1. current four skills: + - `archive-workorder-grid-push-monitor` + - `available-balance-below-zero-monitor` + - `command-center-fee-control-monitor` + - `sgcc-todo-crawler` +2. future new skills without redesigning the dashboard data path +3. low-coupling consumption where dashboard does not parse runtime-private `run-record` structure +4. safe fallback when the latest run fails but the last successful business result should still remain visible + +This design is not about changing business detection logic. +It is about adding a stable result contract and a stable local read path for dashboard consumption. + +## Non-Negotiable Architecture Constraints + +The implementation must preserve these boundaries: + +1. `run-record.json` remains the raw runtime/audit artifact +2. dashboard must not directly consume `run-record.json` +3. normalized result files become the only supported dashboard-facing data contract +4. `claw-new` is responsible for producing normalized files +5. `digital-employee` is responsible for reading normalized files through a local API +6. dashboard is responsible only for rendering and polling the local API +7. first implementation uses HTTP polling, not websocket + +## Why Direct Dashboard Parsing Is Rejected + +The current raw result files are large, runtime-private, and still evolving. + +If `digital-employee` directly parses: + +1. `decisionPreview` +2. `auditPreview.detectReadDiagnostics` +3. skill-specific nested fields + +then the dashboard becomes tightly coupled to runtime internals. + +That would make every runtime refinement risky for UI consumption. + +The dashboard must instead consume a stable, normalized contract that: + +1. preserves business values +2. hides runtime-private structure +3. supports future extractor growth without UI rewrites + +## Why Websocket Is Not The First Transport + +The source data is not user-interactive event flow. +It is scheduled task output that refreshes at minute-level cadence. + +Therefore the first transport must be: + +1. normalized file snapshot +2. local HTTP reader +3. dashboard polling + +This keeps the system simpler and more stable: + +1. no connection state +2. no reconnect logic +3. no push protocol versioning +4. no incremental event semantics + +If true push is needed later, it can be added inside the local reader layer without changing normalized file contract. + +## End-State Architecture + +The full data chain is: + +```text +scheduled skill runtime + -> *.run-record.json + -> normalized result extractor + -> normalized/*.json + -> digital-employee local reader API + -> dashboard polling +``` + +The architecture is intentionally split into four layers. + +### Layer 1: Raw Result Layer + +Location: + +```text +D:\desk\sgclaw\sgclaw\results\*.run-record.json +``` + +Responsibility: + +1. preserve raw runtime result +2. preserve diagnostics/audit structure +3. remain implementation-private to runtime and normalizer + +This layer is allowed to be large and unstable. + +### Layer 2: Normalized Result Layer + +Location: + +```text +D:\desk\sgclaw\sgclaw\results\normalized\ +``` + +Responsibility: + +1. expose stable dashboard-facing contract +2. split one file per skill +3. preserve both latest state and last-known-good state +4. provide index-level aggregation for dashboard first screen + +This is the single consumption contract for downstream readers. + +### Layer 3: Local Reader Layer + +Location: + +```text +D:\data\ideaSpace\rust\sgClaw\digital-employee\server\ +``` + +Responsibility: + +1. read normalized files from disk +2. expose stable local HTTP endpoints +3. validate skill ids and schema shape +4. hide file-path and filesystem concerns from the browser + +This layer does not parse raw `run-record.json`. +It only reads normalized files. + +### Layer 4: Dashboard Consumption Layer + +Location: + +```text +D:\data\ideaSpace\rust\sgClaw\digital-employee\src\ +``` + +Responsibility: + +1. poll local reader API +2. render cards, lists, trend widgets, and detail views +3. react to `status`, `metric`, `payload`, and `freshness` + +This layer must not know anything about raw runtime-private nesting. + +## Required Directory Layout + +The normalized result tree must follow this layout: + +```text +D:\desk\sgclaw\sgclaw\results\ + archive-workorder-grid-push-monitor.run-record.json + available-balance-below-zero-monitor.run-record.json + command-center-fee-control-monitor.run-record.json + sgcc-todo-crawler.run-record.json + normalized\ + index.json + latest\ + archive-workorder-grid-push-monitor.json + available-balance-below-zero-monitor.json + command-center-fee-control-monitor.json + sgcc-todo-crawler.json + last-good\ + archive-workorder-grid-push-monitor.json + available-balance-below-zero-monitor.json + command-center-fee-control-monitor.json + sgcc-todo-crawler.json + history\ + archive-workorder-grid-push-monitor\ + 2026\ + 04\ + 2026-04-25T19-46-06+08-00.json + available-balance-below-zero-monitor\ + command-center-fee-control-monitor\ + sgcc-todo-crawler\ +``` + +Required semantics: + +1. `latest/` always represents most recent normalized extraction +2. `last-good/` updates only when normalized result is still business-usable: + - `ok` + - `empty` + - optionally `soft_error` only if the extractor explicitly certifies business values are still trustworthy +3. `history/` stores immutable snapshots for audit and later trend construction +4. `index.json` summarizes current state of all known normalized skills + +## Normalized Contract + +All normalized files must share one public envelope. + +### Common Envelope + +```json +{ + "schemaVersion": "1.0", + "skillId": "available-balance-below-zero-monitor", + "skillName": "可用电费小于零监测", + "category": "monitor", + "resultType": "count_snapshot", + "observedAt": "2026-04-25T19:46:24+08:00", + "generatedAt": "2026-04-25T19:46:26+08:00", + "status": "ok", + "freshness": { + "staleAfterSeconds": 900, + "isStale": false + }, + "summary": "2026-04-25 19:46:24--可用电费小于零监测检测到【数量】:4265", + "metric": { + "label": "数量", + "value": 4265, + "unit": "items" + }, + "payload": null, + "diagnostics": {}, + "source": { + "kind": "run_record", + "runRecordPath": "D:\\desk\\sgclaw\\sgclaw\\results\\available-balance-below-zero-monitor.run-record.json", + "runRecordMtime": "2026-04-25T19:46:26+08:00", + "extractorVersion": "1.0" + } +} +``` + +### Envelope Field Rules + +#### `schemaVersion` + +This is the public normalized schema version. +It is not the raw runtime version. + +#### `skillId` + +This must equal the stable skill directory / scene id. +It is the primary machine identity. + +#### `skillName` + +This must come from a normalizer registry, not from opportunistic raw JSON text extraction. +This avoids encoding drift and log-text coupling. + +#### `category` + +Current categories: + +1. `monitor` +2. `crawler` + +This is mainly for dashboard grouping and future filtering. + +#### `resultType` + +Current supported public result types: + +1. `count_snapshot` +2. `detail_snapshot` + +New skill onboarding should fit one of these two whenever possible. + +#### `observedAt` + +This is the business observation timestamp. +Priority: + +1. stable run completion / business timestamp from raw result if available +2. raw `run-record.json` file mtime as fallback + +#### `generatedAt` + +This is the time normalized file is written. + +#### `status` + +Allowed values: + +1. `ok` +2. `empty` +3. `soft_error` +4. `error` + +#### `freshness` + +This communicates staleness independent of success/failure. + +Required fields: + +1. `staleAfterSeconds` +2. `isStale` + +The first version may compute `isStale` at generation time or local-reader response time. +Local-reader recomputation is preferred because it reflects current wall-clock state. + +#### `summary` + +This is a stable human-readable sentence assembled by the normalizer from structured fields. +It must never be parsed back by the dashboard. + +#### `metric` + +This is the primary business indicator for card rendering. +All result types must still expose `metric`. + +#### `payload` + +This is `null` for simple count snapshots. +This is a structured object for detail snapshots. + +#### `diagnostics` + +This is public-but-auxiliary. +It is for drill-down, debugging, and support tooling. +Dashboard must not rely on it for primary rendering. + +#### `source` + +This preserves audit linkage back to raw result source. + +Required fields: + +1. `kind` +2. `runRecordPath` +3. `runRecordMtime` +4. `extractorVersion` + +## Result Types + +### Type 1: `count_snapshot` + +This type is used for: + +1. `available-balance-below-zero-monitor` +2. `archive-workorder-grid-push-monitor` +3. `command-center-fee-control-monitor` + +Characteristics: + +1. the business primary output is one main count +2. the dashboard mostly needs a number card and a summary line +3. drill-down diagnostics may exist, but primary payload is count-focused + +Example: + +```json +{ + "schemaVersion": "1.0", + "skillId": "archive-workorder-grid-push-monitor", + "skillName": "归档工单配网推送监测", + "category": "monitor", + "resultType": "count_snapshot", + "observedAt": "2026-04-25T19:46:06+08:00", + "generatedAt": "2026-04-25T19:46:06+08:00", + "status": "soft_error", + "freshness": { + "staleAfterSeconds": 900, + "isStale": false + }, + "summary": "2026-04-25 19:46:06--归档工单配网推送监测检测到【数量】:0", + "metric": { + "label": "数量", + "value": 0, + "unit": "items" + }, + "payload": null, + "diagnostics": { + "pendingCount": 0, + "queryStatus": "soft_error", + "queryError": "XHR 0 for .../getWkorderAll" + }, + "source": { + "kind": "run_record", + "runRecordPath": "D:\\desk\\sgclaw\\sgclaw\\results\\archive-workorder-grid-push-monitor.run-record.json", + "runRecordMtime": "2026-04-25T19:46:06+08:00", + "extractorVersion": "1.0" + } +} +``` + +### Type 2: `detail_snapshot` + +This type is used for: + +1. `sgcc-todo-crawler` + +Characteristics: + +1. the business primary output is a collection of items +2. the dashboard still needs a count for summary cards +3. future downstream consumers may need full item detail + +The public contract must therefore preserve both: + +1. projected stable items for easy UI rendering +2. raw item collection for future richer extraction needs + +Example: + +```json +{ + "schemaVersion": "1.0", + "skillId": "sgcc-todo-crawler", + "skillName": "国网待办抓取", + "category": "crawler", + "resultType": "detail_snapshot", + "observedAt": "2026-04-25T19:47:36+08:00", + "generatedAt": "2026-04-25T19:47:36+08:00", + "status": "ok", + "freshness": { + "staleAfterSeconds": 900, + "isStale": false + }, + "summary": "2026-04-25 19:47:36--国网待办抓取同步到【待办数量】:42", + "metric": { + "label": "待办数量", + "value": 42, + "unit": "items" + }, + "payload": { + "items": [ + { + "id": "todo_8f9b1c", + "index": 1, + "datetime": "2026-04-24 08:25", + "tag": "会议", + "title": "...", + "processNode": "...", + "titleWithProcess": "...", + "user": "...", + "unread": true, + "href": "" + } + ], + "rawItems": [ + { + "datetime": "...", + "href": "", + "index": 1, + "processNode": "...", + "tag": "...", + "title": "...", + "titleWithProcess": "...", + "unread": true, + "user": "..." + } + ], + "aggregates": { + "total": 42, + "unread": 17, + "read": 25 + } + }, + "diagnostics": { + "pendingCount": 42, + "itemSchemaVersion": "1.0" + }, + "source": { + "kind": "run_record", + "runRecordPath": "D:\\desk\\sgclaw\\sgclaw\\results\\sgcc-todo-crawler.run-record.json", + "runRecordMtime": "2026-04-25T19:47:36+08:00", + "extractorVersion": "1.0" + } +} +``` + +## Status Semantics + +Status must not collapse business emptiness and technical failure. + +### `ok` + +Use when: + +1. raw source was readable +2. extractor succeeded +3. business result is trustworthy +4. any auxiliary diagnostics do not undermine the main business output + +### `empty` + +Use when: + +1. the query path succeeded +2. the business result is truly empty +3. `metric.value == 0` +4. there is no technical indication that `0` is merely a failed read + +`empty` is a business conclusion, not a technical fallback. + +### `soft_error` + +Use when: + +1. a normalized result can still be produced +2. but some query stage, partial path, helper path, or side read degraded +3. and the extractor wants that degradation visible to the dashboard or operators + +`soft_error` may still include a usable `metric`. + +### `error` + +Use when: + +1. raw file missing +2. JSON parse failure +3. required business fields missing +4. extractor cannot certify a business result + +In this case `latest/` updates, but `last-good/` must not be replaced. + +## Skill Registry And Mapping Rules + +The normalizer must maintain a registry that defines for each known skill: + +1. `skillId` +2. `skillName` +3. `category` +4. `resultType` +5. summary template +6. extractor implementation binding +7. stale timeout default + +This registry is the only place that should encode display name and primary result classification. + +### Current Skill Registry + +#### `available-balance-below-zero-monitor` + +1. `skillName = "可用电费小于零监测"` +2. `category = "monitor"` +3. `resultType = "count_snapshot"` +4. `summary = "