From d85fae08520dddef9266a66b511a4c82dc7fe49e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?=E7=9F=B3=E5=A4=B4?= <3301352378@qq.com>
Date: Thu, 9 Jul 2026 17:06:50 +0800
Subject: [PATCH] docs: plan N15 selector report hardening

---
 ...elector-report-hardening-implementation.md | 466 ++++++++++++++++++
 1 file changed, 466 insertions(+)
 create mode 100644 docs/superpowers/plans/2026-07-09-n15-selector-report-hardening-implementation.md

diff --git a/docs/superpowers/plans/2026-07-09-n15-selector-report-hardening-implementation.md b/docs/superpowers/plans/2026-07-09-n15-selector-report-hardening-implementation.md
new file mode 100644
index 0000000..322d82f
--- /dev/null
+++ b/docs/superpowers/plans/2026-07-09-n15-selector-report-hardening-implementation.md
@@ -0,0 +1,466 @@
+# N15 Selector Report Hardening Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Harden the N14 offline selector report tests and verifier so ambiguous strict-mode behavior and report-output safety regressions are caught before merge.
+
+**Architecture:** Keep N15 as a test/verifier-only change. Add an in-memory ambiguous-match test to `internal/uiaselector/report_test.go`, then strengthen `scripts/verify-n14-uia-selector-report.ps1` with stale-output cleanup, file/non-empty checks, and JSON match-field allowlist validation.
+
+**Tech Stack:** Go 1.23.x tests, existing `internal/uiaselector` package, Windows PowerShell 5.1-compatible verifier script, standard `go test` and repository Git workflow.
+
+---
+
+## Scope and boundary
+
+N15 changes only test and verifier coverage.
+
+Modify only:
+
+```text
+internal/uiaselector/report_test.go
+scripts/verify-n14-uia-selector-report.ps1
+```
+
+Do not modify:
+
+```text
+native/ISphereWinHelper/Program.cs
+internal/tools/winhelper.go
+cmd/isphere-mcp/main.go
+cmd/uia-selector-report/main.go
+internal/uiaselector/report.go
+internal/uiaselector/catalog.go
+internal/uiaselector/matcher.go
+```
+
+Do not add selectors, helper ops, MCP tools, CLI arguments, live hwnd/process handling, UI automation calls, clicks, keyboard input, clipboard operations, conversation actions, message sending, file transfer, or visible text export.
+
+## File structure
+
+```text
+internal/uiaselector/report_test.go
+  Add one explicit ambiguous strict-mode regression test using an in-memory UIA tree.
+
+scripts/verify-n14-uia-selector-report.ps1
+  Clean stale outputs before CLI execution, require fresh non-empty report files, and validate JSON match fields with an allowlist.
+```
+
+## Task 1: Cover ambiguous strict-mode classification
+
+**Files:**
+- Modify: `internal/uiaselector/report_test.go`
+
+- [ ] **Step 1: Add the ambiguous strict-mode regression test**
+
+Append this test to `internal/uiaselector/report_test.go`:
+
+```go
+func TestStrictModeClassifiesAmbiguousMatches(t *testing.T) {
+	root := &Node{
+		ControlType:  "Window",
+		AutomationID: "frmMain",
+		FrameworkID:  "WinForm",
+		Children: []Node{
+			{ControlType: "Pane", AutomationID: "duplicate", FrameworkID: "WinForm"},
+			{ControlType: "Pane", AutomationID: "duplicate", FrameworkID: "WinForm"},
+		},
+	}
+	catalog := []Selector{
+		{
+			ID:             "duplicate_panel",
+			Description:    "duplicate structural selector",
+			AllowedUse:     AllowedUseReadOnlyLocate,
+			Required:       Criteria{AutomationID: "duplicate", ControlType: "Pane", FrameworkID: "WinForm"},
+			StabilityScore: 80,
+			Evidence:       Evidence{CaptureID: "unit-test", UIAPath: "root/duplicate"},
+		},
+	}
+
+	report := RunCatalogReport("synthetic-ambiguous.json", root, catalog, fixedReportTime())
+	if !report.OK {
+		t.Fatalf("report OK=false: %#v", report.Error)
+	}
+	if report.CatalogSize != 1 {
+		t.Fatalf("catalog_size = %d, want 1", report.CatalogSize)
+	}
+	if report.MatchedCount != 1 {
+		t.Fatalf("matched_count = %d, want 1", report.MatchedCount)
+	}
+	if report.UnmatchedCount != 0 {
+		t.Fatalf("unmatched_count = %d, want 0", report.UnmatchedCount)
+	}
+	if report.AmbiguousCount != 1 {
+		t.Fatalf("ambiguous_count = %d, want 1", report.AmbiguousCount)
+	}
+	if len(report.Selectors) != 1 {
+		t.Fatalf("len(selectors) = %d, want 1", len(report.Selectors))
+	}
+	if got := report.Selectors[0].MatchCount; got != 2 {
+		t.Fatalf("selector match_count = %d, want 2", got)
+	}
+	if got := StrictExitCode(report); got != StrictExitAmbiguous {
+		t.Fatalf("StrictExitCode = %d, want %d", got, StrictExitAmbiguous)
+	}
+}
+```
+
+- [ ] **Step 2: Prove the test catches a broken ambiguous branch**
+
+Temporarily change `internal/uiaselector/report.go` in `StrictExitCode` from:
+
+```go
+	if report.AmbiguousCount > 0 {
+		return StrictExitAmbiguous
+	}
+```
+
+to:
+
+```go
+	if report.AmbiguousCount > 0 {
+		return StrictExitOK
+	}
+```
+
+Run:
+
+```powershell
+gofmt -w internal\uiaselector\report_test.go
+go test ./internal/uiaselector -run "TestStrictModeClassifiesAmbiguousMatches" -count=1
+```
+
+Expected: FAIL with:
+
+```text
+StrictExitCode = 0, want 3
+```
+
+Restore `internal/uiaselector/report.go` immediately after this command so production code again returns `StrictExitAmbiguous` for ambiguous reports. Do not commit the temporary mutation.
+
+- [ ] **Step 3: Run the ambiguous test with production code restored**
+
+Run:
+
+```powershell
+gofmt -w internal\uiaselector\report_test.go internal\uiaselector\report.go
+go test ./internal/uiaselector -run "TestStrictModeClassifiesAmbiguousMatches" -count=1
+```
+
+Expected: PASS.
+
+- [ ] **Step 4: Run package tests**
+
+Run:
+
+```powershell
+go test ./internal/uiaselector -count=1
+```
+
+Expected: PASS.
+
+- [ ] **Step 5: Commit ambiguous strict-mode test**
+
+Run:
+
+```powershell
+git add internal\uiaselector\report_test.go
+git diff --cached --name-only
+```
+
+Expected staged file list:
+
+```text
+internal/uiaselector/report_test.go
+```
+
+Then commit:
+
+```powershell
+git commit -m "test: cover N15 ambiguous strict exit"
+```
+
+## Task 2: Harden N14 selector report verifier
+
+**Files:**
+- Modify: `scripts/verify-n14-uia-selector-report.ps1`
+
+- [ ] **Step 1: Run a failing pre-change verifier hardening check**
+
+Run this PowerShell check before editing the verifier:
+
+```powershell
+$scriptPath = "scripts\verify-n14-uia-selector-report.ps1"
+$scriptText = Get-Content -LiteralPath $scriptPath -Raw -Encoding UTF8
+$requiredSnippets = @(
+    'Remove-Item -LiteralPath $jsonPath, $mdPath -Force -ErrorAction SilentlyContinue',
+    '$jsonItem = Get-Item -LiteralPath $jsonPath -ErrorAction Stop',
+    'if ($jsonItem.PSIsContainer)',
+    'if ($jsonItem.Length -le 0)',
+    '$mdItem = Get-Item -LiteralPath $mdPath -ErrorAction Stop',
+    '$allowedMatchFields = @(',
+    '$match.PSObject.Properties',
+    'JSON match leaks visible text field'
+)
+foreach ($snippet in $requiredSnippets) {
+    if (-not $scriptText.Contains($snippet)) {
+        throw "missing verifier hardening snippet: $snippet"
+    }
+}
+```
+
+Expected: FAIL, with the first missing snippet indicating stale-output cleanup is not implemented yet.
+
+- [ ] **Step 2: Insert stale-output cleanup before CLI execution**
+
+In `scripts/verify-n14-uia-selector-report.ps1`, after:
+
+```powershell
+$jsonPath = Join-Path $outDir "n12r-selector-report.json"
+$mdPath = Join-Path $outDir "n12r-selector-report.md"
+```
+
+insert:
+
+```powershell
+Remove-Item -LiteralPath $jsonPath, $mdPath -Force -ErrorAction SilentlyContinue
+```
+
+- [ ] **Step 3: Replace output file existence checks with file and non-empty checks**
+
+Replace this block:
+
+```powershell
+if (-not (Test-Path -LiteralPath $jsonPath)) {
+    throw "JSON report was not created: $jsonPath"
+}
+if (-not (Test-Path -LiteralPath $mdPath)) {
+    throw "Markdown report was not created: $mdPath"
+}
+```
+
+with:
+
+```powershell
+$jsonItem = Get-Item -LiteralPath $jsonPath -ErrorAction Stop
+if ($jsonItem.PSIsContainer) {
+    throw "JSON report path is not a file: $jsonPath"
+}
+if ($jsonItem.Length -le 0) {
+    throw "JSON report is empty: $jsonPath"
+}
+$mdItem = Get-Item -LiteralPath $mdPath -ErrorAction Stop
+if ($mdItem.PSIsContainer) {
+    throw "Markdown report path is not a file: $mdPath"
+}
+if ($mdItem.Length -le 0) {
+    throw "Markdown report is empty: $mdPath"
+}
+```
+
+- [ ] **Step 4: Add JSON match-field allowlist validation after report summary checks**
+
+After this block:
+
+```powershell
+if ($report.ambiguous_count -ne 0) {
+    throw "ambiguous_count=$($report.ambiguous_count), want 0"
+}
+```
+
+insert:
+
+```powershell
+Write-Host "## JSON match field allowlist"
+$allowedMatchFields = @(
+    "path",
+    "control_type",
+    "automation_id",
+    "class_name",
+    "framework_id",
+    "has_name",
+    "name_length"
+)
+$allowedMatchFieldLookup = @{}
+foreach ($field in $allowedMatchFields) {
+    $allowedMatchFieldLookup[$field] = $true
+}
+foreach ($selector in @($report.selectors)) {
+    foreach ($match in @($selector.matches)) {
+        if ($null -eq $match) {
+            continue
+        }
+        foreach ($property in @($match.PSObject.Properties)) {
+            $fieldName = $property.Name
+            if ($fieldName -eq "name" -or $fieldName -eq "value") {
+                throw "JSON match leaks visible text field: $fieldName"
+            }
+            if (-not $allowedMatchFieldLookup.ContainsKey($fieldName)) {
+                throw "JSON match contains disallowed field: $fieldName"
+            }
+        }
+    }
+}
+```
+
+- [ ] **Step 5: Run the hardening check again**
+
+Run:
+
+```powershell
+$scriptPath = "scripts\verify-n14-uia-selector-report.ps1"
+$scriptText = Get-Content -LiteralPath $scriptPath -Raw -Encoding UTF8
+$requiredSnippets = @(
+    'Remove-Item -LiteralPath $jsonPath, $mdPath -Force -ErrorAction SilentlyContinue',
+    '$jsonItem = Get-Item -LiteralPath $jsonPath -ErrorAction Stop',
+    'if ($jsonItem.PSIsContainer)',
+    'if ($jsonItem.Length -le 0)',
+    '$mdItem = Get-Item -LiteralPath $mdPath -ErrorAction Stop',
+    '$allowedMatchFields = @(',
+    '$match.PSObject.Properties',
+    'JSON match leaks visible text field'
+)
+foreach ($snippet in $requiredSnippets) {
+    if (-not $scriptText.Contains($snippet)) {
+        throw "missing verifier hardening snippet: $snippet"
+    }
+}
+Write-Host "VERIFIER_HARDENING_SNIPPETS_OK"
+```
+
+Expected:
+
+```text
+VERIFIER_HARDENING_SNIPPETS_OK
+```
+
+- [ ] **Step 6: Run the hardened verifier**
+
+Run:
+
+```powershell
+powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-n14-uia-selector-report.ps1
+```
+
+Expected output includes:
+
+```text
+## go test ./internal/uiaselector ./cmd/uia-selector-report
+## go run ./cmd/uia-selector-report
+N14 selector report generated: catalog=10 matched=10 unmatched=0 ambiguous=0
+## JSON match field allowlist
+## output safety scan
+N14 UIA selector report verification passed.
+```
+
+- [ ] **Step 7: Commit verifier hardening**
+
+Run:
+
+```powershell
+git add scripts\verify-n14-uia-selector-report.ps1
+git diff --cached --name-only
+```
+
+Expected staged file list:
+
+```text
+scripts/verify-n14-uia-selector-report.ps1
+```
+
+Then commit:
+
+```powershell
+git commit -m "test: harden N15 selector report verifier"
+```
+
+## Task 3: Final verification and boundary check
+
+**Files:**
+- No new files.
+- Verify: `internal/uiaselector/report_test.go`, `scripts/verify-n14-uia-selector-report.ps1`, and unchanged helper/MCP action surface.
+
+- [ ] **Step 1: Run targeted tests**
+
+Run:
+
+```powershell
+go test ./internal/uiaselector -run "TestStrictModeClassifiesAmbiguousMatches" -count=1
+go test ./internal/uiaselector ./cmd/uia-selector-report -count=1
+```
+
+Expected: both commands PASS.
+
+- [ ] **Step 2: Run repository tests**
+
+Run:
+
+```powershell
+go test ./...
+```
+
+Expected: all packages PASS.
+
+If dependency download fails with a transient proxy error, retry once with:
+
+```powershell
+$env:GOPROXY = "https://goproxy.cn,direct"
+go test ./...
+```
+
+- [ ] **Step 3: Run the hardened verifier**
+
+Run:
+
+```powershell
+powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-n14-uia-selector-report.ps1
+```
+
+Expected: verifier PASS and output includes `## JSON match field allowlist`.
+
+- [ ] **Step 4: Check helper/MCP/live action boundary**
+
+Run:
+
+```powershell
+git diff --name-only main..HEAD -- native/ISphereWinHelper/Program.cs internal/tools/winhelper.go cmd/isphere-mcp/main.go cmd/uia-selector-report/main.go internal/uiaselector/report.go internal/uiaselector/catalog.go internal/uiaselector/matcher.go
+```
+
+Expected: no output.
+
+If any file appears, stop and inspect because N15 is test/verifier-only.
+
+- [ ] **Step 5: Check changed files**
+
+Run:
+
+```powershell
+git diff --name-only main..HEAD
+```
+
+Expected changed files are limited to:
+
+```text
+docs/superpowers/specs/2026-07-09-n15-selector-report-hardening-design.md
+docs/superpowers/plans/2026-07-09-n15-selector-report-hardening-implementation.md
+internal/uiaselector/report_test.go
+scripts/verify-n14-uia-selector-report.ps1
+```
+
+- [ ] **Step 6: Final status**
+
+Run:
+
+```powershell
+git status --short --branch
+git log --oneline --max-count=6
+```
+
+Expected: working tree clean on `codex/n15-report-hardening`. Do not push unless the user explicitly requests pushing this branch or merging to main.
+
+## Self-review checklist
+
+- Spec coverage: ambiguous strict-mode test, stale-output cleanup, file/non-empty checks, JSON match-field allowlist, retained forbidden string scan, and helper/MCP boundary check are all covered.
+- Scope: only `internal/uiaselector/report_test.go` and `scripts/verify-n14-uia-selector-report.ps1` are implementation targets; design and plan docs are documentation targets.
+- Boundary: no helper op, no MCP tool, no live hwnd/process behavior, no UI action, no report format change, no CLI argument change.
+- Verification: final commands include targeted tests, all repository tests, hardened verifier, boundary diff check, and changed-file check.
+- Commit cadence: test and verifier hardening are separate commits.
\ No newline at end of file