28 KiB
Skill Normalized Result And Dashboard Local Reader Design
Date: 2026-04-26
Status: Proposed design for implementation
Related baseline designs:
docs/superpowers/specs/2026-04-22-scheduled-monitoring-action-end-to-end-architecture-design.mddocs/superpowers/specs/2026-04-23-archive-workorder-grid-push-monitor-design.mddocs/superpowers/specs/2026-04-24-available-balance-below-zero-monitor-design.mddocs/superpowers/specs/2026-04-26-scheduled-monitoring-generated-scene-adaptation-hardening-design.md
Related skill/result sources:
D:/desk/sgclaw/sgclaw/results/archive-workorder-grid-push-monitor.run-record.jsonD:/desk/sgclaw/sgclaw/results/available-balance-below-zero-monitor.run-record.jsonD:/desk/sgclaw/sgclaw/results/command-center-fee-control-monitor.run-record.jsonD:/desk/sgclaw/sgclaw/results/sgcc-todo-crawler.run-record.json
Target dashboard project:
D:/data/ideaSpace/rust/sgClaw/digital-employee
Plain-Language Goal
Turn raw scheduled-skill run results into a stable, dashboard-friendly result contract.
The system must support:
- current four skills:
archive-workorder-grid-push-monitoravailable-balance-below-zero-monitorcommand-center-fee-control-monitorsgcc-todo-crawler
- future new skills without redesigning the dashboard data path
- low-coupling consumption where dashboard does not parse runtime-private
run-recordstructure - safe fallback when the latest run fails but the last successful business result should still remain visible
This design is not about changing business detection logic. It is about adding a stable result contract and a stable local read path for dashboard consumption.
Non-Negotiable Architecture Constraints
The implementation must preserve these boundaries:
run-record.jsonremains the raw runtime/audit artifact- dashboard must not directly consume
run-record.json - normalized result files become the only supported dashboard-facing data contract
claw-newis responsible for producing normalized filesdigital-employeeis responsible for reading normalized files through a local API- dashboard is responsible only for rendering and polling the local API
- first implementation uses HTTP polling, not websocket
Why Direct Dashboard Parsing Is Rejected
The current raw result files are large, runtime-private, and still evolving.
If digital-employee directly parses:
decisionPreviewauditPreview.detectReadDiagnostics- skill-specific nested fields
then the dashboard becomes tightly coupled to runtime internals.
That would make every runtime refinement risky for UI consumption.
The dashboard must instead consume a stable, normalized contract that:
- preserves business values
- hides runtime-private structure
- supports future extractor growth without UI rewrites
Why Websocket Is Not The First Transport
The source data is not user-interactive event flow. It is scheduled task output that refreshes at minute-level cadence.
Therefore the first transport must be:
- normalized file snapshot
- local HTTP reader
- dashboard polling
This keeps the system simpler and more stable:
- no connection state
- no reconnect logic
- no push protocol versioning
- no incremental event semantics
If true push is needed later, it can be added inside the local reader layer without changing normalized file contract.
End-State Architecture
The full data chain is:
scheduled skill runtime
-> *.run-record.json
-> normalized result extractor
-> normalized/*.json
-> digital-employee local reader API
-> dashboard polling
The architecture is intentionally split into four layers.
Layer 1: Raw Result Layer
Location:
D:\desk\sgclaw\sgclaw\results\*.run-record.json
Responsibility:
- preserve raw runtime result
- preserve diagnostics/audit structure
- remain implementation-private to runtime and normalizer
This layer is allowed to be large and unstable.
Layer 2: Normalized Result Layer
Location:
D:\desk\sgclaw\sgclaw\results\normalized\
Responsibility:
- expose stable dashboard-facing contract
- split one file per skill
- preserve both latest state and last-known-good state
- provide index-level aggregation for dashboard first screen
This is the single consumption contract for downstream readers.
Layer 3: Local Reader Layer
Location:
D:\data\ideaSpace\rust\sgClaw\digital-employee\server\
Responsibility:
- read normalized files from disk
- expose stable local HTTP endpoints
- validate skill ids and schema shape
- hide file-path and filesystem concerns from the browser
This layer does not parse raw run-record.json.
It only reads normalized files.
Layer 4: Dashboard Consumption Layer
Location:
D:\data\ideaSpace\rust\sgClaw\digital-employee\src\
Responsibility:
- poll local reader API
- render cards, lists, trend widgets, and detail views
- react to
status,metric,payload, andfreshness
This layer must not know anything about raw runtime-private nesting.
Required Directory Layout
The normalized result tree must follow this layout:
D:\desk\sgclaw\sgclaw\results\
archive-workorder-grid-push-monitor.run-record.json
available-balance-below-zero-monitor.run-record.json
command-center-fee-control-monitor.run-record.json
sgcc-todo-crawler.run-record.json
normalized\
index.json
latest\
archive-workorder-grid-push-monitor.json
available-balance-below-zero-monitor.json
command-center-fee-control-monitor.json
sgcc-todo-crawler.json
last-good\
archive-workorder-grid-push-monitor.json
available-balance-below-zero-monitor.json
command-center-fee-control-monitor.json
sgcc-todo-crawler.json
history\
archive-workorder-grid-push-monitor\
2026\
04\
2026-04-25T19-46-06+08-00.json
available-balance-below-zero-monitor\
command-center-fee-control-monitor\
sgcc-todo-crawler\
Required semantics:
latest/always represents most recent normalized extractionlast-good/updates only when normalized result is still business-usable:okempty- optionally
soft_erroronly if the extractor explicitly certifies business values are still trustworthy
history/stores immutable snapshots for audit and later trend constructionindex.jsonsummarizes current state of all known normalized skills
Normalized Contract
All normalized files must share one public envelope.
Common Envelope
{
"schemaVersion": "1.0",
"skillId": "available-balance-below-zero-monitor",
"skillName": "可用电费小于零监测",
"category": "monitor",
"resultType": "count_snapshot",
"observedAt": "2026-04-25T19:46:24+08:00",
"generatedAt": "2026-04-25T19:46:26+08:00",
"status": "ok",
"freshness": {
"staleAfterSeconds": 900,
"isStale": false
},
"summary": "2026-04-25 19:46:24--可用电费小于零监测检测到【数量】:4265",
"metric": {
"label": "数量",
"value": 4265,
"unit": "items"
},
"payload": null,
"diagnostics": {},
"source": {
"kind": "run_record",
"runRecordPath": "D:\\desk\\sgclaw\\sgclaw\\results\\available-balance-below-zero-monitor.run-record.json",
"runRecordMtime": "2026-04-25T19:46:26+08:00",
"extractorVersion": "1.0"
}
}
Envelope Field Rules
schemaVersion
This is the public normalized schema version. It is not the raw runtime version.
skillId
This must equal the stable skill directory / scene id. It is the primary machine identity.
skillName
This must come from a normalizer registry, not from opportunistic raw JSON text extraction. This avoids encoding drift and log-text coupling.
category
Current categories:
monitorcrawler
This is mainly for dashboard grouping and future filtering.
resultType
Current supported public result types:
count_snapshotdetail_snapshot
New skill onboarding should fit one of these two whenever possible.
observedAt
This is the business observation timestamp. Priority:
- stable run completion / business timestamp from raw result if available
- raw
run-record.jsonfile mtime as fallback
generatedAt
This is the time normalized file is written.
status
Allowed values:
okemptysoft_errorerror
freshness
This communicates staleness independent of success/failure.
Required fields:
staleAfterSecondsisStale
The first version may compute isStale at generation time or local-reader response time.
Local-reader recomputation is preferred because it reflects current wall-clock state.
summary
This is a stable human-readable sentence assembled by the normalizer from structured fields. It must never be parsed back by the dashboard.
metric
This is the primary business indicator for card rendering.
All result types must still expose metric.
payload
This is null for simple count snapshots.
This is a structured object for detail snapshots.
diagnostics
This is public-but-auxiliary. It is for drill-down, debugging, and support tooling. Dashboard must not rely on it for primary rendering.
source
This preserves audit linkage back to raw result source.
Required fields:
kindrunRecordPathrunRecordMtimeextractorVersion
Result Types
Type 1: count_snapshot
This type is used for:
available-balance-below-zero-monitorarchive-workorder-grid-push-monitorcommand-center-fee-control-monitor
Characteristics:
- the business primary output is one main count
- the dashboard mostly needs a number card and a summary line
- drill-down diagnostics may exist, but primary payload is count-focused
Example:
{
"schemaVersion": "1.0",
"skillId": "archive-workorder-grid-push-monitor",
"skillName": "归档工单配网推送监测",
"category": "monitor",
"resultType": "count_snapshot",
"observedAt": "2026-04-25T19:46:06+08:00",
"generatedAt": "2026-04-25T19:46:06+08:00",
"status": "soft_error",
"freshness": {
"staleAfterSeconds": 900,
"isStale": false
},
"summary": "2026-04-25 19:46:06--归档工单配网推送监测检测到【数量】:0",
"metric": {
"label": "数量",
"value": 0,
"unit": "items"
},
"payload": null,
"diagnostics": {
"pendingCount": 0,
"queryStatus": "soft_error",
"queryError": "XHR 0 for .../getWkorderAll"
},
"source": {
"kind": "run_record",
"runRecordPath": "D:\\desk\\sgclaw\\sgclaw\\results\\archive-workorder-grid-push-monitor.run-record.json",
"runRecordMtime": "2026-04-25T19:46:06+08:00",
"extractorVersion": "1.0"
}
}
Type 2: detail_snapshot
This type is used for:
sgcc-todo-crawler
Characteristics:
- the business primary output is a collection of items
- the dashboard still needs a count for summary cards
- future downstream consumers may need full item detail
The public contract must therefore preserve both:
- projected stable items for easy UI rendering
- raw item collection for future richer extraction needs
Example:
{
"schemaVersion": "1.0",
"skillId": "sgcc-todo-crawler",
"skillName": "国网待办抓取",
"category": "crawler",
"resultType": "detail_snapshot",
"observedAt": "2026-04-25T19:47:36+08:00",
"generatedAt": "2026-04-25T19:47:36+08:00",
"status": "ok",
"freshness": {
"staleAfterSeconds": 900,
"isStale": false
},
"summary": "2026-04-25 19:47:36--国网待办抓取同步到【待办数量】:42",
"metric": {
"label": "待办数量",
"value": 42,
"unit": "items"
},
"payload": {
"items": [
{
"id": "todo_8f9b1c",
"index": 1,
"datetime": "2026-04-24 08:25",
"tag": "会议",
"title": "...",
"processNode": "...",
"titleWithProcess": "...",
"user": "...",
"unread": true,
"href": ""
}
],
"rawItems": [
{
"datetime": "...",
"href": "",
"index": 1,
"processNode": "...",
"tag": "...",
"title": "...",
"titleWithProcess": "...",
"unread": true,
"user": "..."
}
],
"aggregates": {
"total": 42,
"unread": 17,
"read": 25
}
},
"diagnostics": {
"pendingCount": 42,
"itemSchemaVersion": "1.0"
},
"source": {
"kind": "run_record",
"runRecordPath": "D:\\desk\\sgclaw\\sgclaw\\results\\sgcc-todo-crawler.run-record.json",
"runRecordMtime": "2026-04-25T19:47:36+08:00",
"extractorVersion": "1.0"
}
}
Status Semantics
Status must not collapse business emptiness and technical failure.
ok
Use when:
- raw source was readable
- extractor succeeded
- business result is trustworthy
- any auxiliary diagnostics do not undermine the main business output
empty
Use when:
- the query path succeeded
- the business result is truly empty
metric.value == 0- there is no technical indication that
0is merely a failed read
empty is a business conclusion, not a technical fallback.
soft_error
Use when:
- a normalized result can still be produced
- but some query stage, partial path, helper path, or side read degraded
- and the extractor wants that degradation visible to the dashboard or operators
soft_error may still include a usable metric.
error
Use when:
- raw file missing
- JSON parse failure
- required business fields missing
- extractor cannot certify a business result
In this case latest/ updates, but last-good/ must not be replaced.
Skill Registry And Mapping Rules
The normalizer must maintain a registry that defines for each known skill:
skillIdskillNamecategoryresultType- summary template
- extractor implementation binding
- stale timeout default
This registry is the only place that should encode display name and primary result classification.
Current Skill Registry
available-balance-below-zero-monitor
skillName = "可用电费小于零监测"category = "monitor"resultType = "count_snapshot"summary = "<time>--可用电费小于零监测检测到【数量】:<value>"
Primary count source priority:
auditPreview.detectReadDiagnostics.rawMergedCountdecisionPreview.summary.pending_countdecisionPreview.pendingList.length
Primary diagnostics to preserve:
slice01Countslice02Countslice03CountqueriedSlicessliceErrorsrequestTimeoutMsreadStepTraces
Soft-error conditions:
sliceErrorsnon-empty- any
readStepTraces.status != "ok" - missing required slice with fallback count still present
archive-workorder-grid-push-monitor
skillName = "归档工单配网推送监测"category = "monitor"resultType = "count_snapshot"summary = "<time>--归档工单配网推送监测检测到【数量】:<value>"
Primary count source priority:
auditPreview.detectReadDiagnostics.newItemCountauditPreview.detectReadDiagnostics.filteredCountdecisionPreview.summary.pending_countdecisionPreview.pendingList.length
Primary diagnostics to preserve:
rawCountfilteredCountdedupedCountnewItemCountqueryStatusqueryErrorrequestParamreadStepTraces
Soft-error conditions:
queryStatus != "ok"- any
readStepTraces.status == "soft_error" - extractor still able to produce a count but read quality degraded
command-center-fee-control-monitor
skillName = "指挥中心费控异常监测"category = "monitor"resultType = "count_snapshot"summary = "<time>--指挥中心费控异常监测检测到【数量】:<value>"
Primary count source priority:
auditPreview.detectReadDiagnostics.queryAbnorListCountdecisionPreview.summary.pending_countdecisionPreview.pendingList.length
Primary diagnostics to preserve:
queryAbnorListCountqueryHistoryEnergyChargeCountgetOrgTreeStatusgetMonitorLogStatusgetOtherIphonesStatusreadStepTraces
Soft-error conditions:
- primary abnormal-list query degraded
- supporting reads degraded in a way that should remain operator-visible
- business count remains present but read quality is partial
sgcc-todo-crawler
skillName = "国网待办抓取"category = "crawler"resultType = "detail_snapshot"summary = "<time>--国网待办抓取同步到【待办数量】:<value>"
Primary count source priority:
decisionPreview.pendingList.length
Primary payload source:
decisionPreview.pendingList
Projected item fields:
idindexdatetimetagtitleprocessNodetitleWithProcessuserunreadhref
The extractor must produce:
payload.itemspayload.rawItemspayload.aggregates
Why sgcc-todo-crawler Needs Both items And rawItems
The user requirement is not only to display a count. Future dashboard or downstream logic may need to read the larger business JSON.
Directly exposing raw run-record is rejected.
Instead the normalized detail snapshot must preserve:
- stable projected items for common UI use
- raw source items for future richer access
This gives:
- low coupling for normal UI
- no future data loss
- no need for the dashboard to parse runtime-private envelopes
index.json Contract
The dashboard first screen must not fetch every skill individually. It should start from one summary index.
Example:
{
"schemaVersion": "1.0",
"generatedAt": "2026-04-25T19:48:00+08:00",
"revision": "2026-04-25T19:48:00+08:00",
"skills": [
{
"skillId": "available-balance-below-zero-monitor",
"skillName": "可用电费小于零监测",
"category": "monitor",
"resultType": "count_snapshot",
"status": "ok",
"observedAt": "2026-04-25T19:46:24+08:00",
"metric": {
"label": "数量",
"value": 4265,
"unit": "items"
},
"summary": "2026-04-25 19:46:24--可用电费小于零监测检测到【数量】:4265",
"currentFile": "latest/available-balance-below-zero-monitor.json",
"lastGoodFile": "last-good/available-balance-below-zero-monitor.json"
}
]
}
Required properties:
schemaVersiongeneratedAtrevisionskills[]
The index must contain enough information for:
- top-level dashboard cards
- quick health/status summary
- later detail-file lookup
claw-new Normalizer Design
The normalizer must live as a first-class module, not ad hoc file-writing logic.
Recommended structure:
src/result_normalization/
mod.rs
contract.rs
registry.rs
extractors/
available_balance.rs
archive_workorder.rs
command_center_fee_control.rs
sgcc_todo_crawler.rs
writer.rs
index_builder.rs
contract.rs
Defines public normalized structures:
- envelope
- count snapshot payload model
- detail snapshot payload model
- index model
registry.rs
Defines:
- skill ids
- display names
- result types
- stale defaults
- extractor binding
extractors/*
Each extractor:
- reads one raw
run-record - validates required fields
- derives business result
- derives public diagnostics
- assigns
status - produces normalized contract
Extractors must not write files directly.
writer.rs
Responsible for:
- writing
latest/ - conditionally updating
last-good/ - writing immutable
history/ - atomic replace semantics
index_builder.rs
Reads generated normalized skill outputs and rebuilds index.json.
claw-new CLI / Trigger Entry
The first implementation must support both:
- on-demand local backfill after copying raw results
- future automatic generation at runtime completion
Required CLI shape:
sg_claw normalize-results --results-dir <path>
sg_claw normalize-results --results-dir <path> --skills skill1,skill2
Why this is required:
- current workflow copies raw results from the inner network to local machine
- the design cannot assume automatic end-to-end generation exists on day one
- manual backfill keeps current operational workflow usable
File Write And Atomicity Rules
The normalizer must never write directly into final targets without staging.
Required write protocol:
- write
*.tmp - fsync/flush if practical
- rename over final target
Write order:
- skill
latest/ - skill
last-good/if eligible - skill
history/ - rebuild
index.jsonlast
This prevents dashboard from reading partially-written JSON.
last-good Update Policy
The policy must be explicit.
Update last-good when:
- status is
ok - status is
empty - status is
soft_erroronly if the extractor explicitly marks the business value as trustworthy
Do not update last-good when:
- status is
error - extractor cannot certify the primary metric
This protects dashboard against losing prior valid business state because of one bad latest run.
digital-employee Local Reader Design
The local reader must be embedded inside digital-employee, not implemented as a separate external service.
Recommended structure:
D:\data\ideaSpace\rust\sgClaw\digital-employee\
server\
index.js
config.js
routes\
results.js
services\
fileRepository.js
resultStore.js
validators.js
Recommended technology:
- Node
- Express or a minimal native HTTP wrapper
It must:
- bind only to
127.0.0.1 - expose only read-only APIs
- validate skill ids
- read only normalized result files
- never expose arbitrary path traversal
The local reader must not:
- parse raw
run-record.json - expose raw result directory as static file root
- implement business transformation logic
Local Reader API Contract
Required endpoints:
GET /api/health
Returns:
- local reader status
- configured results dir
- configured normalized dir
- current known revision
GET /api/results
Returns:
- parsed
index.json
Used by:
- dashboard first load
- periodic summary polling
GET /api/results/:skillId
Returns:
currentlastGood
This supports:
- current health/status display
- fallback display when latest is degraded
GET /api/results/:skillId/history?limit=30
Returns:
- most recent N history snapshots
This is for:
- future trends
- drill-down panels
- audit-friendly exploration
GET /api/results/:skillId/items
Valid only for detail_snapshot.
Returns:
- projected
payload.items
GET /api/results/:skillId/raw-items
Valid only for detail_snapshot.
Returns:
payload.rawItems
This endpoint exists because future consumers may need the larger business JSON without depending on raw run-record envelope.
Polling Strategy
First version uses polling only.
Recommended behavior:
- dashboard polls
GET /api/resultsevery 30 seconds - dashboard fetches per-skill detail lazily when the relevant card/detail view opens
- no websocket
- no browser filesystem access
This is sufficient because source data is minute-scale scheduled output.
Dashboard Consumption Model
The dashboard must become index-driven, not hardcoded by private source structure.
Current static data files such as:
src/data/work-reports.jsonsrc/data/anomaly-logs.json
are implementation placeholders and must not remain the live results source for these skills.
Recommended dashboard data structure:
src/
api/
results.js
store/
modules/
results.js
The UI should consume:
- top-level card data from
/api/results - detail view data from
/api/results/:skillId - list/table panels from
/api/results/:skillId/items
Dashboard rendering must branch only on:
resultTypestatusmetricfreshness
It must not branch on raw runtime field names.
New Skill Onboarding Contract
Future new skills should reuse the same path.
To onboard a new skill:
- add registry entry in
claw-new - add extractor
- emit
count_snapshotordetail_snapshot - rebuild
index.json
If the skill fits one of those two result types, then:
- normalized writer does not need redesign
- local reader does not need redesign
- dashboard data access layer does not need redesign
Only if a truly new output shape appears should a new resultType be added.
Current Workflow Compatibility
The design must support the current operational workflow:
- run inside the inner network
- copy raw
run-record.jsonartifacts to local machine - locally normalize those copied files
- serve normalized files into
digital-employee
Therefore the first implementation must support this exact path:
copy raw results
-> run local normalize command
-> generate normalized/
-> local reader serves normalized/
-> dashboard polls local reader
This avoids blocking the design on immediate upstream runtime automation.
Error And Fallback Behavior
The dashboard must not go blank simply because the latest run degraded.
Therefore the local reader must expose:
currentlastGood
Dashboard may then display:
- latest status badge from
current.status - latest observed time from
current.observedAt - fallback metric from
lastGood.metricwhencurrent.status == "error"
This is the core resilience feature of the design.
Staleness Rules
Staleness is independent of status.
A result may be:
okbut staleemptybut stalesoft_errorand stale
This is why freshness exists as a separate field group.
Default stale threshold for first version:
- 900 seconds
This default may be overridden per skill in the registry if needed later.
Security Constraints
The local reader must obey these restrictions:
- bind only to localhost
- expose no write endpoints
- expose no arbitrary filesystem browsing
- expose only normalized result content
- validate skill ids against registry/index
The normalizer must obey:
- no destructive modification of raw
run-recordartifacts - no in-place partial overwrite without temp file + rename
Configuration
Required configurable values:
SGCLAW_RESULTS_DIRSGCLAW_NORMALIZED_DIRLOCAL_READER_HOSTLOCAL_READER_PORTRESULT_STALE_AFTER_SECONDS
Suggested defaults for local workflow:
SGCLAW_RESULTS_DIR = D:\desk\sgclaw\sgclaw\resultsSGCLAW_NORMALIZED_DIR = D:\desk\sgclaw\sgclaw\results\normalizedLOCAL_READER_HOST = 127.0.0.1LOCAL_READER_PORT = 31337RESULT_STALE_AFTER_SECONDS = 900
Testing Scope
claw-new Normalizer Tests
Required test categories:
- fixture-based extractor tests for all four current skills
- file-writing tests for
latest,last-good,history index.jsonbuild tests- malformed raw file tests
- missing raw file tests
soft_errorpreservation tests0count but successful-empty tests
digital-employee Local Reader Tests
Required test categories:
/api/health/api/results/api/results/:skillId/api/results/:skillId/history/api/results/:skillId/items/api/results/:skillId/raw-items- unknown skill rejection
- missing normalized file handling
Dashboard Data-Layer Tests
Required test categories:
- initial results load
- polling refresh
current.status == "error"withlastGoodfallbackdetail_snapshotlist rendering- stale marker rendering
Out Of Scope For This Design
This design does not define:
- final visual layout of the dashboard cards
- websocket push transport
- modification of original monitoring business logic
- active side effects
- inner-network deployment automation
Those may be handled by later plans.
Final Design Summary
The final design deliberately creates one stable seam:
- raw runtime output stays raw
claw-newproduces normalized business resultsdigital-employeereads only normalized files through a local API- dashboard consumes only local HTTP contract
This is the minimum architecture that gives:
- high cohesion
- low coupling
- current-workflow compatibility
- support for the existing four skills
- extensibility for future new skills