feat: add generated scene skill platform hardening
This commit is contained in:
@@ -0,0 +1,130 @@
|
||||
# Structured Fail-Closed Improvement Roadmap Design
|
||||
|
||||
> Date: 2026-04-19
|
||||
> Status: Draft
|
||||
> Upstream Reconciliation: `tests/fixtures/generated_scene/full_sweep_status_reconciliation_2026-04-19.json`
|
||||
|
||||
## Intent
|
||||
|
||||
Turn the `48` structured fail-closed records from the reconciled `102` sweep into a governed improvement roadmap.
|
||||
|
||||
The objective is not to weaken gates or inflate `auto-pass`. The objective is to classify contract gaps, identify the highest-value bounded correction slices, and then improve generic scene-to-skill conversion where evidence can be recovered safely.
|
||||
|
||||
## Current Reconciled Baseline
|
||||
|
||||
After status reconciliation, the `102` scene set is:
|
||||
|
||||
| Reconciled status | Count |
|
||||
| --- | ---: |
|
||||
| `auto-pass` | 48 |
|
||||
| `fail-closed-known` | 48 |
|
||||
| `adjudicated-valid-host-bridge` | 4 |
|
||||
| `source-unreadable` | 2 |
|
||||
| `missing-source` | 0 |
|
||||
| `unsupported-family` | 0 |
|
||||
|
||||
The `4` raw route conflicts are no longer unresolved route bugs. They are valid host-bridge workflows.
|
||||
|
||||
This roadmap therefore focuses on `fail-closed-known = 48`.
|
||||
|
||||
## Fail-Closed Buckets
|
||||
|
||||
| Inferred archetype | Reason | Count |
|
||||
| --- | --- | ---: |
|
||||
| `paginated_enrichment` | `workflow evidence is incomplete before package generation` | 35 |
|
||||
| `local_doc_pipeline` | `workflow evidence is incomplete before package generation` | 5 |
|
||||
| `multi_mode_request` | `workflow evidence is incomplete before package generation` | 4 |
|
||||
| `single_request_enrichment` | `workflow evidence is incomplete before package generation` | 2 |
|
||||
| `host_bridge_workflow` | `workflow evidence is incomplete before package generation` | 1 |
|
||||
| `page_state_eval` | `bootstrap_target` | 1 |
|
||||
|
||||
The first priority is the `35` `paginated_enrichment` records because they are the largest bucket and map to the most important currently generic workflow family.
|
||||
|
||||
## Scope Guardrails
|
||||
|
||||
In scope:
|
||||
|
||||
1. classify the `48` structured fail-closed records by missing contract piece
|
||||
2. prioritize bounded correction slices
|
||||
3. implement bounded evidence recovery only after classification shows repeated recoverable patterns
|
||||
4. keep all fail-closed semantics intact
|
||||
5. rerun a bounded follow-up sweep after corrections
|
||||
|
||||
Out of scope:
|
||||
|
||||
1. adding new scene families
|
||||
2. starting `G4/G5`
|
||||
3. login recovery
|
||||
4. full browser host runtime transport
|
||||
5. local document attachment runtime
|
||||
6. auto-promoting scenes into the execution board
|
||||
7. weakening readiness gates to increase pass counts
|
||||
8. reopening the already adjudicated `4` valid-host-bridge workflows
|
||||
9. handling the `2` remaining timeout records in this roadmap
|
||||
|
||||
## Workstreams
|
||||
|
||||
1. `WS1` Fail-Closed Inventory and Gap Taxonomy
|
||||
2. `WS2` G3 Paginated Enrichment Contract Recovery
|
||||
3. `WS3` Small-Bucket Contract Recovery
|
||||
4. `WS4` Bootstrap Target Isolation
|
||||
5. `WS5` Follow-Up Sweep and Coverage Delta
|
||||
|
||||
## Gap Taxonomy
|
||||
|
||||
Every structured fail-closed record must receive one primary missing-contract label:
|
||||
|
||||
1. `main_request_missing`
|
||||
2. `pagination_plan_missing`
|
||||
3. `enrichment_request_missing`
|
||||
4. `join_key_missing`
|
||||
5. `export_plan_missing`
|
||||
6. `mode_matrix_missing`
|
||||
7. `mode_request_contract_missing`
|
||||
8. `single_request_enrichment_contract_missing`
|
||||
9. `host_bridge_contract_missing`
|
||||
10. `local_doc_contract_missing`
|
||||
11. `bootstrap_target_unresolved`
|
||||
12. `mixed_or_ambiguous_contract_gap`
|
||||
|
||||
Secondary labels may be added, but every record must have exactly one primary label.
|
||||
|
||||
## Correction Strategy
|
||||
|
||||
Corrections must be pattern-based, not scene-by-scene.
|
||||
|
||||
Allowed correction types:
|
||||
|
||||
1. bounded evidence extraction for repeated field names or workflow structures
|
||||
2. bounded IR fallback only when evidence is explicit and traceable
|
||||
3. more specific fail-closed reason reporting
|
||||
4. regression tests for each recovered pattern
|
||||
|
||||
Forbidden correction types:
|
||||
|
||||
1. hard-coding a scene name to pass
|
||||
2. converting fail-closed records to pass without closing the contract
|
||||
3. broad route-precedence rewrites
|
||||
4. disabling or relaxing gates
|
||||
|
||||
## Expected Outputs
|
||||
|
||||
1. `tests/fixtures/generated_scene/structured_fail_closed_inventory_2026-04-19.json`
|
||||
2. `tests/fixtures/generated_scene/structured_fail_closed_improvement_followup_2026-04-19.json`
|
||||
3. `docs/superpowers/reports/2026-04-19-structured-fail-closed-inventory-report.md`
|
||||
4. `docs/superpowers/reports/2026-04-19-structured-fail-closed-improvement-coverage-delta-report.md`
|
||||
5. `docs/superpowers/reports/2026-04-19-structured-fail-closed-improvement-roadmap-closure-report.md`
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
1. all `48` fail-closed records are inventoried
|
||||
2. all `48` records have exactly one primary missing-contract label
|
||||
3. the `35` `paginated_enrichment` records are split into actionable G3 gap groups
|
||||
4. implementation, if performed, is limited to repeated recoverable patterns
|
||||
5. no adjudicated host-bridge record is reopened
|
||||
6. follow-up results are measured against the reconciled baseline
|
||||
7. execution board status remains unchanged
|
||||
|
||||
## Completion Signal
|
||||
|
||||
The roadmap is complete when the `48` structured fail-closed records are no longer a single broad bucket and the follow-up sweep quantifies whether bounded evidence recovery improved safe conversion coverage.
|
||||
Reference in New Issue
Block a user