131 lines
5.0 KiB
Markdown
131 lines
5.0 KiB
Markdown
# Structured Fail-Closed Improvement Roadmap Design
|
|
|
|
> Date: 2026-04-19
|
|
> Status: Draft
|
|
> Upstream Reconciliation: `tests/fixtures/generated_scene/full_sweep_status_reconciliation_2026-04-19.json`
|
|
|
|
## Intent
|
|
|
|
Turn the `48` structured fail-closed records from the reconciled `102` sweep into a governed improvement roadmap.
|
|
|
|
The objective is not to weaken gates or inflate `auto-pass`. The objective is to classify contract gaps, identify the highest-value bounded correction slices, and then improve generic scene-to-skill conversion where evidence can be recovered safely.
|
|
|
|
## Current Reconciled Baseline
|
|
|
|
After status reconciliation, the `102` scene set is:
|
|
|
|
| Reconciled status | Count |
|
|
| --- | ---: |
|
|
| `auto-pass` | 48 |
|
|
| `fail-closed-known` | 48 |
|
|
| `adjudicated-valid-host-bridge` | 4 |
|
|
| `source-unreadable` | 2 |
|
|
| `missing-source` | 0 |
|
|
| `unsupported-family` | 0 |
|
|
|
|
The `4` raw route conflicts are no longer unresolved route bugs. They are valid host-bridge workflows.
|
|
|
|
This roadmap therefore focuses on `fail-closed-known = 48`.
|
|
|
|
## Fail-Closed Buckets
|
|
|
|
| Inferred archetype | Reason | Count |
|
|
| --- | --- | ---: |
|
|
| `paginated_enrichment` | `workflow evidence is incomplete before package generation` | 35 |
|
|
| `local_doc_pipeline` | `workflow evidence is incomplete before package generation` | 5 |
|
|
| `multi_mode_request` | `workflow evidence is incomplete before package generation` | 4 |
|
|
| `single_request_enrichment` | `workflow evidence is incomplete before package generation` | 2 |
|
|
| `host_bridge_workflow` | `workflow evidence is incomplete before package generation` | 1 |
|
|
| `page_state_eval` | `bootstrap_target` | 1 |
|
|
|
|
The first priority is the `35` `paginated_enrichment` records because they are the largest bucket and map to the most important currently generic workflow family.
|
|
|
|
## Scope Guardrails
|
|
|
|
In scope:
|
|
|
|
1. classify the `48` structured fail-closed records by missing contract piece
|
|
2. prioritize bounded correction slices
|
|
3. implement bounded evidence recovery only after classification shows repeated recoverable patterns
|
|
4. keep all fail-closed semantics intact
|
|
5. rerun a bounded follow-up sweep after corrections
|
|
|
|
Out of scope:
|
|
|
|
1. adding new scene families
|
|
2. starting `G4/G5`
|
|
3. login recovery
|
|
4. full browser host runtime transport
|
|
5. local document attachment runtime
|
|
6. auto-promoting scenes into the execution board
|
|
7. weakening readiness gates to increase pass counts
|
|
8. reopening the already adjudicated `4` valid-host-bridge workflows
|
|
9. handling the `2` remaining timeout records in this roadmap
|
|
|
|
## Workstreams
|
|
|
|
1. `WS1` Fail-Closed Inventory and Gap Taxonomy
|
|
2. `WS2` G3 Paginated Enrichment Contract Recovery
|
|
3. `WS3` Small-Bucket Contract Recovery
|
|
4. `WS4` Bootstrap Target Isolation
|
|
5. `WS5` Follow-Up Sweep and Coverage Delta
|
|
|
|
## Gap Taxonomy
|
|
|
|
Every structured fail-closed record must receive one primary missing-contract label:
|
|
|
|
1. `main_request_missing`
|
|
2. `pagination_plan_missing`
|
|
3. `enrichment_request_missing`
|
|
4. `join_key_missing`
|
|
5. `export_plan_missing`
|
|
6. `mode_matrix_missing`
|
|
7. `mode_request_contract_missing`
|
|
8. `single_request_enrichment_contract_missing`
|
|
9. `host_bridge_contract_missing`
|
|
10. `local_doc_contract_missing`
|
|
11. `bootstrap_target_unresolved`
|
|
12. `mixed_or_ambiguous_contract_gap`
|
|
|
|
Secondary labels may be added, but every record must have exactly one primary label.
|
|
|
|
## Correction Strategy
|
|
|
|
Corrections must be pattern-based, not scene-by-scene.
|
|
|
|
Allowed correction types:
|
|
|
|
1. bounded evidence extraction for repeated field names or workflow structures
|
|
2. bounded IR fallback only when evidence is explicit and traceable
|
|
3. more specific fail-closed reason reporting
|
|
4. regression tests for each recovered pattern
|
|
|
|
Forbidden correction types:
|
|
|
|
1. hard-coding a scene name to pass
|
|
2. converting fail-closed records to pass without closing the contract
|
|
3. broad route-precedence rewrites
|
|
4. disabling or relaxing gates
|
|
|
|
## Expected Outputs
|
|
|
|
1. `tests/fixtures/generated_scene/structured_fail_closed_inventory_2026-04-19.json`
|
|
2. `tests/fixtures/generated_scene/structured_fail_closed_improvement_followup_2026-04-19.json`
|
|
3. `docs/superpowers/reports/2026-04-19-structured-fail-closed-inventory-report.md`
|
|
4. `docs/superpowers/reports/2026-04-19-structured-fail-closed-improvement-coverage-delta-report.md`
|
|
5. `docs/superpowers/reports/2026-04-19-structured-fail-closed-improvement-roadmap-closure-report.md`
|
|
|
|
## Acceptance Criteria
|
|
|
|
1. all `48` fail-closed records are inventoried
|
|
2. all `48` records have exactly one primary missing-contract label
|
|
3. the `35` `paginated_enrichment` records are split into actionable G3 gap groups
|
|
4. implementation, if performed, is limited to repeated recoverable patterns
|
|
5. no adjudicated host-bridge record is reopened
|
|
6. follow-up results are measured against the reconciled baseline
|
|
7. execution board status remains unchanged
|
|
|
|
## Completion Signal
|
|
|
|
The roadmap is complete when the `48` structured fail-closed records are no longer a single broad bucket and the follow-up sweep quantifies whether bounded evidence recovery improved safe conversion coverage.
|