907 lines
27 KiB
Markdown
907 lines
27 KiB
Markdown
# N14 UIA Selector Report Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Build an offline N14 selector report generator that reads a UIA dump file, runs the N13 selector catalog, and writes safe JSON/Markdown reports without adding live helper or MCP actions.
|
|
|
|
**Architecture:** Extend `internal/uiaselector` with a focused report model, catalog report generator, strict-mode classification, and JSON/Markdown renderers. Add a small `cmd/uia-selector-report` CLI that only accepts dump-file paths and writes reports; PowerShell verifier wraps tests and CLI execution.
|
|
|
|
**Tech Stack:** Go 1.23.x, standard library `encoding/json`, `flag`, `time`, `os`, existing `internal/uiaselector` package, PowerShell verification wrapper.
|
|
|
|
---
|
|
|
|
## Scope and boundary
|
|
|
|
This plan implements only offline report generation.
|
|
|
|
Do not modify these files:
|
|
|
|
```text
|
|
native/ISphereWinHelper/Program.cs
|
|
internal/tools/winhelper.go
|
|
cmd/isphere-mcp/main.go
|
|
```
|
|
|
|
Do not add helper ops, MCP tools, live window handles, process discovery, UI automation API calls, clicks, keyboard input, clipboard actions, conversation actions, message sending, file transfer, or visible text export.
|
|
|
|
Every selector remains constrained to `read_only_locate`. Reports must not output full UIA `name` or `value` fields. Reports may output only derived text-safety fields already used by N13:
|
|
|
|
```text
|
|
has_name
|
|
name_length
|
|
```
|
|
|
|
## File structure
|
|
|
|
Create these files:
|
|
|
|
```text
|
|
internal/uiaselector/report_test.go
|
|
internal/uiaselector/report.go
|
|
cmd/uia-selector-report/main_test.go
|
|
cmd/uia-selector-report/main.go
|
|
scripts/verify-n14-uia-selector-report.ps1
|
|
```
|
|
|
|
Responsibilities:
|
|
|
|
- `internal/uiaselector/report.go`: report data structures, catalog report generation, strict exit classification, JSON rendering/writing, Markdown rendering/writing.
|
|
- `internal/uiaselector/report_test.go`: unit tests for catalog reporting, strict classification, and no-visible-text report rendering.
|
|
- `cmd/uia-selector-report/main.go`: CLI argument parsing and file output orchestration.
|
|
- `cmd/uia-selector-report/main_test.go`: CLI tests using temporary output files and the N12R redacted fixture.
|
|
- `scripts/verify-n14-uia-selector-report.ps1`: Go test + CLI smoke verification + output safety scans.
|
|
|
|
## Task 1: Add report model, generator, and strict-mode classification
|
|
|
|
**Files:**
|
|
- Create: `internal/uiaselector/report_test.go`
|
|
- Create: `internal/uiaselector/report.go`
|
|
|
|
- [ ] **Step 1: Write failing report generator tests**
|
|
|
|
Create `internal/uiaselector/report_test.go`:
|
|
|
|
```go
|
|
package uiaselector
|
|
|
|
import (
|
|
"testing"
|
|
"time"
|
|
)
|
|
|
|
func fixedReportTime() time.Time {
|
|
return time.Date(2026, 7, 9, 0, 0, 0, 0, time.UTC)
|
|
}
|
|
|
|
func loadN14Fixture(t *testing.T) *Node {
|
|
t.Helper()
|
|
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
|
if err != nil {
|
|
t.Fatalf("LoadDumpFile: %v", err)
|
|
}
|
|
return root
|
|
}
|
|
|
|
func TestRunCatalogReportMatchesAllN13Selectors(t *testing.T) {
|
|
report := RunCatalogReport(
|
|
"testdata/n12r-2026-07-09-uia-redacted.json",
|
|
loadN14Fixture(t),
|
|
DefaultCatalog(),
|
|
fixedReportTime(),
|
|
)
|
|
if !report.OK {
|
|
t.Fatalf("report OK=false: %#v", report.Error)
|
|
}
|
|
if report.SourceDump != "testdata/n12r-2026-07-09-uia-redacted.json" {
|
|
t.Fatalf("source_dump = %q", report.SourceDump)
|
|
}
|
|
if report.GeneratedAtUTC != "2026-07-09T00:00:00Z" {
|
|
t.Fatalf("generated_at_utc = %q", report.GeneratedAtUTC)
|
|
}
|
|
if report.CatalogSize != 10 {
|
|
t.Fatalf("catalog_size = %d, want 10", report.CatalogSize)
|
|
}
|
|
if report.MatchedCount != 10 {
|
|
t.Fatalf("matched_count = %d, want 10", report.MatchedCount)
|
|
}
|
|
if report.UnmatchedCount != 0 {
|
|
t.Fatalf("unmatched_count = %d, want 0", report.UnmatchedCount)
|
|
}
|
|
if report.AmbiguousCount != 0 {
|
|
t.Fatalf("ambiguous_count = %d, want 0", report.AmbiguousCount)
|
|
}
|
|
if got := StrictExitCode(report); got != StrictExitOK {
|
|
t.Fatalf("StrictExitCode = %d, want %d", got, StrictExitOK)
|
|
}
|
|
}
|
|
|
|
func TestStrictModeClassifiesFailures(t *testing.T) {
|
|
catalog := []Selector{
|
|
{
|
|
ID: "missing_panel",
|
|
Description: "missing structural test selector",
|
|
AllowedUse: AllowedUseReadOnlyLocate,
|
|
Required: Criteria{AutomationID: "does_not_exist", ControlType: "Pane", FrameworkID: "WinForm"},
|
|
StabilityScore: 80,
|
|
Evidence: Evidence{CaptureID: "unit-test", UIAPath: "root/missing"},
|
|
},
|
|
}
|
|
report := RunCatalogReport("fixture.json", loadN14Fixture(t), catalog, fixedReportTime())
|
|
if !report.OK {
|
|
t.Fatalf("report OK=false: %#v", report.Error)
|
|
}
|
|
if report.UnmatchedCount != 1 {
|
|
t.Fatalf("unmatched_count = %d, want 1", report.UnmatchedCount)
|
|
}
|
|
if got := StrictExitCode(report); got != StrictExitUnmatched {
|
|
t.Fatalf("StrictExitCode = %d, want %d", got, StrictExitUnmatched)
|
|
}
|
|
}
|
|
|
|
func TestReportRejectsNonReadOnlyCatalogEntries(t *testing.T) {
|
|
catalog := []Selector{
|
|
{
|
|
ID: "bad_write_selector",
|
|
Description: "invalid write selector",
|
|
AllowedUse: "write",
|
|
Required: Criteria{AutomationID: "frmMain", ControlType: "Window", FrameworkID: "WinForm"},
|
|
StabilityScore: 1,
|
|
Evidence: Evidence{CaptureID: "unit-test", UIAPath: "root"},
|
|
},
|
|
}
|
|
report := RunCatalogReport("fixture.json", loadN14Fixture(t), catalog, fixedReportTime())
|
|
if report.OK {
|
|
t.Fatalf("report OK=true for invalid catalog")
|
|
}
|
|
if report.Error == nil || report.Error.Code != "INVALID_CATALOG" {
|
|
t.Fatalf("report error = %#v, want INVALID_CATALOG", report.Error)
|
|
}
|
|
if got := StrictExitCode(report); got != StrictExitInvalidCatalog {
|
|
t.Fatalf("StrictExitCode = %d, want %d", got, StrictExitInvalidCatalog)
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run report generator tests and verify they fail**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
go test ./internal/uiaselector -run "TestRunCatalogReport|TestStrictMode|TestReportRejects" -count=1
|
|
```
|
|
|
|
Expected: FAIL with compile errors for undefined symbols such as `RunCatalogReport`, `StrictExitCode`, and `StrictExitOK`.
|
|
|
|
If `go` is unavailable, stop and install or expose Go 1.23.x before continuing.
|
|
|
|
- [ ] **Step 3: Implement report model, generator, and strict exit classification**
|
|
|
|
Create `internal/uiaselector/report.go`:
|
|
|
|
```go
|
|
package uiaselector
|
|
|
|
import "time"
|
|
|
|
const (
|
|
StrictExitOK = 0
|
|
StrictExitReportGenerationFailure = 1
|
|
StrictExitUnmatched = 2
|
|
StrictExitAmbiguous = 3
|
|
StrictExitInvalidCatalog = 4
|
|
)
|
|
|
|
type Report struct {
|
|
OK bool `json:"ok"`
|
|
SourceDump string `json:"source_dump"`
|
|
CatalogSize int `json:"catalog_size"`
|
|
MatchedCount int `json:"matched_count"`
|
|
UnmatchedCount int `json:"unmatched_count"`
|
|
AmbiguousCount int `json:"ambiguous_count"`
|
|
GeneratedAtUTC string `json:"generated_at_utc"`
|
|
Selectors []SelectorReport `json:"selectors"`
|
|
Warnings []string `json:"warnings"`
|
|
Error *ReportError `json:"error,omitempty"`
|
|
}
|
|
|
|
type ReportError struct {
|
|
Code string `json:"code"`
|
|
Message string `json:"message"`
|
|
}
|
|
|
|
type SelectorReport struct {
|
|
SelectorID string `json:"selector_id"`
|
|
Description string `json:"description"`
|
|
AllowedUse string `json:"allowed_use"`
|
|
StabilityScore int `json:"stability_score"`
|
|
Evidence Evidence `json:"evidence"`
|
|
Matched bool `json:"matched"`
|
|
MatchCount int `json:"match_count"`
|
|
Matches []NodeMatch `json:"matches"`
|
|
Warnings []string `json:"warnings"`
|
|
}
|
|
|
|
func RunCatalogReport(sourceDump string, root *Node, catalog []Selector, generatedAt time.Time) Report {
|
|
report := Report{
|
|
OK: true,
|
|
SourceDump: sourceDump,
|
|
CatalogSize: len(catalog),
|
|
GeneratedAtUTC: generatedAt.UTC().Format(time.RFC3339),
|
|
Warnings: []string{},
|
|
}
|
|
if root == nil {
|
|
report.OK = false
|
|
report.Error = &ReportError{Code: "NO_ROOT", Message: "UIA dump root is nil"}
|
|
return report
|
|
}
|
|
|
|
for _, selector := range catalog {
|
|
if selector.AllowedUse != AllowedUseReadOnlyLocate {
|
|
report.OK = false
|
|
report.Error = &ReportError{Code: "INVALID_CATALOG", Message: "selector is not read-only: " + selector.ID}
|
|
return report
|
|
}
|
|
match := MatchSelector(root, catalog, selector.ID)
|
|
selectorReport := SelectorReport{
|
|
SelectorID: selector.ID,
|
|
Description: selector.Description,
|
|
AllowedUse: selector.AllowedUse,
|
|
StabilityScore: selector.StabilityScore,
|
|
Evidence: selector.Evidence,
|
|
Matched: match.Matched,
|
|
MatchCount: match.MatchCount,
|
|
Matches: match.Matches,
|
|
Warnings: append([]string{}, match.Warnings...),
|
|
}
|
|
report.Selectors = append(report.Selectors, selectorReport)
|
|
if match.Matched {
|
|
report.MatchedCount++
|
|
} else {
|
|
report.UnmatchedCount++
|
|
}
|
|
if match.MatchCount > 1 {
|
|
report.AmbiguousCount++
|
|
}
|
|
}
|
|
return report
|
|
}
|
|
|
|
func StrictExitCode(report Report) int {
|
|
if !report.OK {
|
|
if report.Error != nil && report.Error.Code == "INVALID_CATALOG" {
|
|
return StrictExitInvalidCatalog
|
|
}
|
|
return StrictExitReportGenerationFailure
|
|
}
|
|
if report.UnmatchedCount > 0 {
|
|
return StrictExitUnmatched
|
|
}
|
|
if report.AmbiguousCount > 0 {
|
|
return StrictExitAmbiguous
|
|
}
|
|
return StrictExitOK
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run report generator tests and verify they pass**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
gofmt -w internal\uiaselector\report.go internal\uiaselector\report_test.go
|
|
go test ./internal/uiaselector -run "TestRunCatalogReport|TestStrictMode|TestReportRejects" -count=1
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Commit report generator**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add internal\uiaselector\report.go internal\uiaselector\report_test.go
|
|
git commit -m "feat: add N14 selector report model"
|
|
```
|
|
|
|
## Task 2: Add JSON and Markdown report rendering
|
|
|
|
**Files:**
|
|
- Modify: `internal/uiaselector/report_test.go`
|
|
- Modify: `internal/uiaselector/report.go`
|
|
|
|
- [ ] **Step 1: Write failing rendering tests**
|
|
|
|
Append these tests to `internal/uiaselector/report_test.go`:
|
|
|
|
```go
|
|
func TestReportDoesNotExposeVisibleText(t *testing.T) {
|
|
report := RunCatalogReport(
|
|
"testdata/n12r-2026-07-09-uia-redacted.json",
|
|
loadN14Fixture(t),
|
|
DefaultCatalog(),
|
|
fixedReportTime(),
|
|
)
|
|
body, err := RenderReportJSON(report)
|
|
if err != nil {
|
|
t.Fatalf("RenderReportJSON: %v", err)
|
|
}
|
|
jsonText := string(body)
|
|
for _, forbidden := range []string{`"name"`, `"value"`, "国网甘肃省电力公司"} {
|
|
if strings.Contains(jsonText, forbidden) {
|
|
t.Fatalf("JSON report exposed forbidden text %q in %s", forbidden, jsonText)
|
|
}
|
|
}
|
|
for _, required := range []string{`"has_name"`, `"name_length"`, `"selector_id"`, `"automation_id"`} {
|
|
if !strings.Contains(jsonText, required) {
|
|
t.Fatalf("JSON report missing required structural field %q in %s", required, jsonText)
|
|
}
|
|
}
|
|
}
|
|
|
|
func TestMarkdownReportDoesNotExposeVisibleText(t *testing.T) {
|
|
report := RunCatalogReport(
|
|
"testdata/n12r-2026-07-09-uia-redacted.json",
|
|
loadN14Fixture(t),
|
|
DefaultCatalog(),
|
|
fixedReportTime(),
|
|
)
|
|
markdown := RenderReportMarkdown(report)
|
|
for _, forbidden := range []string{"国网甘肃省电力公司", "| name |", "| value |"} {
|
|
if strings.Contains(markdown, forbidden) {
|
|
t.Fatalf("Markdown report exposed forbidden text %q in %s", forbidden, markdown)
|
|
}
|
|
}
|
|
for _, required := range []string{"# N14 UIA Selector Report", "## Summary", "automation_id", "control_type", "class_name", "has_name", "name_length"} {
|
|
if !strings.Contains(markdown, required) {
|
|
t.Fatalf("Markdown report missing %q in %s", required, markdown)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Add `strings` to the import block so the file begins with:
|
|
|
|
```go
|
|
import (
|
|
"strings"
|
|
"testing"
|
|
"time"
|
|
)
|
|
```
|
|
|
|
- [ ] **Step 2: Run rendering tests and verify they fail**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
go test ./internal/uiaselector -run "TestReportDoesNotExposeVisibleText|TestMarkdownReportDoesNotExposeVisibleText" -count=1
|
|
```
|
|
|
|
Expected: FAIL with compile errors for undefined `RenderReportJSON` and `RenderReportMarkdown`.
|
|
|
|
- [ ] **Step 3: Implement JSON and Markdown rendering/writing**
|
|
|
|
Append this code to `internal/uiaselector/report.go`:
|
|
|
|
```go
|
|
func RenderReportJSON(report Report) ([]byte, error) {
|
|
return json.MarshalIndent(report, "", " ")
|
|
}
|
|
|
|
func WriteReportJSON(report Report, path string) error {
|
|
body, err := RenderReportJSON(report)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
return writeFileWithParents(path, body)
|
|
}
|
|
|
|
func RenderReportMarkdown(report Report) string {
|
|
var b strings.Builder
|
|
b.WriteString("# N14 UIA Selector Report\n\n")
|
|
b.WriteString("- Source dump: `" + escapeMarkdownInline(report.SourceDump) + "`\n")
|
|
b.WriteString(fmt.Sprintf("- Catalog size: %d\n", report.CatalogSize))
|
|
b.WriteString(fmt.Sprintf("- Matched: %d\n", report.MatchedCount))
|
|
b.WriteString(fmt.Sprintf("- Unmatched: %d\n", report.UnmatchedCount))
|
|
b.WriteString(fmt.Sprintf("- Ambiguous: %d\n", report.AmbiguousCount))
|
|
b.WriteString("\n## Summary\n\n")
|
|
b.WriteString("| selector_id | status | matches | stability | evidence_path |\n")
|
|
b.WriteString("| --- | --- | ---: | ---: | --- |\n")
|
|
for _, selector := range report.Selectors {
|
|
status := "unmatched"
|
|
if selector.MatchCount > 1 {
|
|
status = "ambiguous"
|
|
} else if selector.Matched {
|
|
status = "matched"
|
|
}
|
|
b.WriteString(fmt.Sprintf(
|
|
"| %s | %s | %d | %d | %s |\n",
|
|
escapeMarkdownCell(selector.SelectorID),
|
|
status,
|
|
selector.MatchCount,
|
|
selector.StabilityScore,
|
|
escapeMarkdownCell(selector.Evidence.UIAPath),
|
|
))
|
|
}
|
|
b.WriteString("\n## Selector Details\n")
|
|
for _, selector := range report.Selectors {
|
|
b.WriteString("\n### " + escapeMarkdownInline(selector.SelectorID) + "\n\n")
|
|
b.WriteString("- Description: " + escapeMarkdownInline(selector.Description) + "\n")
|
|
b.WriteString("- Allowed use: " + escapeMarkdownInline(selector.AllowedUse) + "\n")
|
|
b.WriteString(fmt.Sprintf("- Stability score: %d\n", selector.StabilityScore))
|
|
b.WriteString(fmt.Sprintf("- Match count: %d\n", selector.MatchCount))
|
|
b.WriteString("\nMatches:\n\n")
|
|
b.WriteString("| path | control_type | automation_id | class_name | framework_id | has_name | name_length |\n")
|
|
b.WriteString("| --- | --- | --- | --- | --- | --- | ---: |\n")
|
|
for _, match := range selector.Matches {
|
|
b.WriteString(fmt.Sprintf(
|
|
"| %s | %s | %s | %s | %s | %t | %d |\n",
|
|
escapeMarkdownCell(match.Path),
|
|
escapeMarkdownCell(match.ControlType),
|
|
escapeMarkdownCell(match.AutomationID),
|
|
escapeMarkdownCell(match.ClassName),
|
|
escapeMarkdownCell(match.FrameworkID),
|
|
match.HasName,
|
|
match.NameLength,
|
|
))
|
|
}
|
|
}
|
|
return b.String()
|
|
}
|
|
|
|
func WriteReportMarkdown(report Report, path string) error {
|
|
return writeFileWithParents(path, []byte(RenderReportMarkdown(report)))
|
|
}
|
|
|
|
func writeFileWithParents(path string, body []byte) error {
|
|
dir := filepath.Dir(path)
|
|
if dir != "." && dir != "" {
|
|
if err := os.MkdirAll(dir, 0o755); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
return os.WriteFile(path, body, 0o644)
|
|
}
|
|
|
|
func escapeMarkdownInline(value string) string {
|
|
return strings.NewReplacer("`", "'", "\r", " ", "\n", " ").Replace(value)
|
|
}
|
|
|
|
func escapeMarkdownCell(value string) string {
|
|
return strings.NewReplacer("|", "\\|", "\r", " ", "\n", " ").Replace(value)
|
|
}
|
|
```
|
|
|
|
Update the import block in `internal/uiaselector/report.go` to:
|
|
|
|
```go
|
|
import (
|
|
"encoding/json"
|
|
"fmt"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"time"
|
|
)
|
|
```
|
|
|
|
- [ ] **Step 4: Run rendering tests and verify they pass**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
gofmt -w internal\uiaselector\report.go internal\uiaselector\report_test.go
|
|
go test ./internal/uiaselector -run "TestReportDoesNotExposeVisibleText|TestMarkdownReportDoesNotExposeVisibleText" -count=1
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Run package tests and commit rendering**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
go test ./internal/uiaselector -count=1
|
|
git add internal\uiaselector\report.go internal\uiaselector\report_test.go
|
|
git commit -m "feat: render N14 selector reports"
|
|
```
|
|
|
|
Expected: package tests PASS before commit.
|
|
|
|
## Task 3: Add offline selector report CLI
|
|
|
|
**Files:**
|
|
- Create: `cmd/uia-selector-report/main_test.go`
|
|
- Create: `cmd/uia-selector-report/main.go`
|
|
|
|
- [ ] **Step 1: Write failing CLI tests**
|
|
|
|
Create `cmd/uia-selector-report/main_test.go`:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"bytes"
|
|
"encoding/json"
|
|
"os"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
func TestRunWritesJSONAndMarkdownReports(t *testing.T) {
|
|
tmp := t.TempDir()
|
|
jsonPath := filepath.Join(tmp, "report.json")
|
|
mdPath := filepath.Join(tmp, "report.md")
|
|
var stdout bytes.Buffer
|
|
var stderr bytes.Buffer
|
|
code := Run([]string{
|
|
"-dump", "../../internal/uiaselector/testdata/n12r-2026-07-09-uia-redacted.json",
|
|
"-out-json", jsonPath,
|
|
"-out-md", mdPath,
|
|
"-strict",
|
|
}, &stdout, &stderr)
|
|
if code != 0 {
|
|
t.Fatalf("Run exit code = %d, stderr=%s stdout=%s", code, stderr.String(), stdout.String())
|
|
}
|
|
jsonBody, err := os.ReadFile(jsonPath)
|
|
if err != nil {
|
|
t.Fatalf("read JSON report: %v", err)
|
|
}
|
|
var parsed struct {
|
|
OK bool `json:"ok"`
|
|
CatalogSize int `json:"catalog_size"`
|
|
MatchedCount int `json:"matched_count"`
|
|
UnmatchedCount int `json:"unmatched_count"`
|
|
AmbiguousCount int `json:"ambiguous_count"`
|
|
}
|
|
if err := json.Unmarshal(jsonBody, &parsed); err != nil {
|
|
t.Fatalf("JSON report did not parse: %v", err)
|
|
}
|
|
if !parsed.OK || parsed.CatalogSize != 10 || parsed.MatchedCount != 10 || parsed.UnmatchedCount != 0 || parsed.AmbiguousCount != 0 {
|
|
t.Fatalf("unexpected JSON report summary: %#v", parsed)
|
|
}
|
|
markdown, err := os.ReadFile(mdPath)
|
|
if err != nil {
|
|
t.Fatalf("read Markdown report: %v", err)
|
|
}
|
|
if !strings.Contains(string(markdown), "## Summary") {
|
|
t.Fatalf("Markdown report missing Summary: %s", string(markdown))
|
|
}
|
|
}
|
|
|
|
func TestRunRequiresDumpPath(t *testing.T) {
|
|
var stdout bytes.Buffer
|
|
var stderr bytes.Buffer
|
|
code := Run([]string{"-strict"}, &stdout, &stderr)
|
|
if code != 1 {
|
|
t.Fatalf("Run exit code = %d, want 1", code)
|
|
}
|
|
if !strings.Contains(stderr.String(), "missing -dump") {
|
|
t.Fatalf("stderr missing dump message: %s", stderr.String())
|
|
}
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run CLI tests and verify they fail**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
go test ./cmd/uia-selector-report -count=1
|
|
```
|
|
|
|
Expected: FAIL because package or `Run` does not exist.
|
|
|
|
- [ ] **Step 3: Implement CLI**
|
|
|
|
Create `cmd/uia-selector-report/main.go`:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"flag"
|
|
"fmt"
|
|
"io"
|
|
"os"
|
|
"time"
|
|
|
|
"isphere-ai-bridge/internal/uiaselector"
|
|
)
|
|
|
|
func main() {
|
|
os.Exit(Run(os.Args[1:], os.Stdout, os.Stderr))
|
|
}
|
|
|
|
func Run(args []string, stdout io.Writer, stderr io.Writer) int {
|
|
flags := flag.NewFlagSet("uia-selector-report", flag.ContinueOnError)
|
|
flags.SetOutput(stderr)
|
|
dumpPath := flags.String("dump", "", "UIA dump JSON path")
|
|
outJSON := flags.String("out-json", "", "JSON report output path")
|
|
outMD := flags.String("out-md", "", "Markdown report output path")
|
|
strict := flags.Bool("strict", false, "return non-zero for unmatched, ambiguous, or invalid selector reports")
|
|
if err := flags.Parse(args); err != nil {
|
|
return uiaselector.StrictExitReportGenerationFailure
|
|
}
|
|
if *dumpPath == "" {
|
|
fmt.Fprintln(stderr, "missing -dump")
|
|
return uiaselector.StrictExitReportGenerationFailure
|
|
}
|
|
|
|
root, err := uiaselector.LoadDumpFile(*dumpPath)
|
|
if err != nil {
|
|
fmt.Fprintf(stderr, "load dump: %v\n", err)
|
|
return uiaselector.StrictExitReportGenerationFailure
|
|
}
|
|
report := uiaselector.RunCatalogReport(*dumpPath, root, uiaselector.DefaultCatalog(), time.Now().UTC())
|
|
if *outJSON != "" {
|
|
if err := uiaselector.WriteReportJSON(report, *outJSON); err != nil {
|
|
fmt.Fprintf(stderr, "write JSON report: %v\n", err)
|
|
return uiaselector.StrictExitReportGenerationFailure
|
|
}
|
|
}
|
|
if *outMD != "" {
|
|
if err := uiaselector.WriteReportMarkdown(report, *outMD); err != nil {
|
|
fmt.Fprintf(stderr, "write Markdown report: %v\n", err)
|
|
return uiaselector.StrictExitReportGenerationFailure
|
|
}
|
|
}
|
|
|
|
code := uiaselector.StrictExitCode(report)
|
|
if *strict && code != uiaselector.StrictExitOK {
|
|
fmt.Fprintf(stderr, "strict report check failed with code %d\n", code)
|
|
return code
|
|
}
|
|
if !report.OK {
|
|
fmt.Fprintf(stderr, "report generation failed: %#v\n", report.Error)
|
|
return uiaselector.StrictExitReportGenerationFailure
|
|
}
|
|
fmt.Fprintf(stdout, "N14 selector report generated: catalog=%d matched=%d unmatched=%d ambiguous=%d\n", report.CatalogSize, report.MatchedCount, report.UnmatchedCount, report.AmbiguousCount)
|
|
return uiaselector.StrictExitOK
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run CLI tests and verify they pass**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
gofmt -w cmd\uia-selector-report\main.go cmd\uia-selector-report\main_test.go
|
|
go test ./cmd/uia-selector-report -count=1
|
|
```
|
|
|
|
Expected: PASS.
|
|
|
|
- [ ] **Step 5: Run relevant tests and commit CLI**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
go test ./internal/uiaselector ./cmd/uia-selector-report -count=1
|
|
git add cmd\uia-selector-report\main.go cmd\uia-selector-report\main_test.go
|
|
git commit -m "feat: add N14 selector report CLI"
|
|
```
|
|
|
|
Expected: both packages PASS before commit.
|
|
|
|
## Task 4: Add N14 verification script
|
|
|
|
**Files:**
|
|
- Create: `scripts/verify-n14-uia-selector-report.ps1`
|
|
|
|
- [ ] **Step 1: Create verification script**
|
|
|
|
Create `scripts/verify-n14-uia-selector-report.ps1`:
|
|
|
|
```powershell
|
|
$ErrorActionPreference = "Stop"
|
|
Set-StrictMode -Version Latest
|
|
|
|
$repo = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path
|
|
Set-Location -LiteralPath $repo
|
|
|
|
if (-not (Get-Command go -ErrorAction SilentlyContinue)) {
|
|
throw "go was not found. Run this verifier in a Go 1.23.x environment."
|
|
}
|
|
|
|
Write-Host "## go test ./internal/uiaselector ./cmd/uia-selector-report"
|
|
go test ./internal/uiaselector ./cmd/uia-selector-report -count=1
|
|
if ($LASTEXITCODE -ne 0) {
|
|
throw "go tests failed with exit code $LASTEXITCODE"
|
|
}
|
|
|
|
$outDir = Join-Path $repo "runs\n14-selector-report"
|
|
New-Item -ItemType Directory -Force -Path $outDir | Out-Null
|
|
$jsonPath = Join-Path $outDir "n12r-selector-report.json"
|
|
$mdPath = Join-Path $outDir "n12r-selector-report.md"
|
|
|
|
Write-Host "## go run ./cmd/uia-selector-report"
|
|
go run ./cmd/uia-selector-report `
|
|
-dump internal/uiaselector/testdata/n12r-2026-07-09-uia-redacted.json `
|
|
-out-json $jsonPath `
|
|
-out-md $mdPath `
|
|
-strict
|
|
if ($LASTEXITCODE -ne 0) {
|
|
throw "uia-selector-report failed with exit code $LASTEXITCODE"
|
|
}
|
|
|
|
if (-not (Test-Path -LiteralPath $jsonPath)) {
|
|
throw "JSON report was not created: $jsonPath"
|
|
}
|
|
if (-not (Test-Path -LiteralPath $mdPath)) {
|
|
throw "Markdown report was not created: $mdPath"
|
|
}
|
|
|
|
$report = Get-Content -LiteralPath $jsonPath -Raw -Encoding UTF8 | ConvertFrom-Json
|
|
if (-not $report.ok) {
|
|
throw "report ok=false"
|
|
}
|
|
if ($report.catalog_size -ne 10) {
|
|
throw "catalog_size=$($report.catalog_size), want 10"
|
|
}
|
|
if ($report.matched_count -ne 10) {
|
|
throw "matched_count=$($report.matched_count), want 10"
|
|
}
|
|
if ($report.unmatched_count -ne 0) {
|
|
throw "unmatched_count=$($report.unmatched_count), want 0"
|
|
}
|
|
if ($report.ambiguous_count -ne 0) {
|
|
throw "ambiguous_count=$($report.ambiguous_count), want 0"
|
|
}
|
|
|
|
Write-Host "## output safety scan"
|
|
$jsonRaw = Get-Content -LiteralPath $jsonPath -Raw -Encoding UTF8
|
|
$mdRaw = Get-Content -LiteralPath $mdPath -Raw -Encoding UTF8
|
|
$forbiddenOutput = @(
|
|
'"name"',
|
|
'"value"',
|
|
'国网甘肃省电力公司',
|
|
'send_after_approval',
|
|
'send_file_after_approval',
|
|
'receive_file_after_approval',
|
|
'open_conversation',
|
|
'InvokePattern',
|
|
'ValuePattern.SetValue'
|
|
)
|
|
foreach ($term in $forbiddenOutput) {
|
|
if ($jsonRaw.Contains($term)) {
|
|
throw "JSON report contains forbidden term: $term"
|
|
}
|
|
if ($mdRaw.Contains($term)) {
|
|
throw "Markdown report contains forbidden term: $term"
|
|
}
|
|
}
|
|
|
|
Write-Host "N14 UIA selector report verification passed."
|
|
```
|
|
|
|
- [ ] **Step 2: Run verifier and verify it passes**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-n14-uia-selector-report.ps1
|
|
```
|
|
|
|
Expected:
|
|
|
|
```text
|
|
## go test ./internal/uiaselector ./cmd/uia-selector-report
|
|
ok isphere-ai-bridge/internal/uiaselector
|
|
ok isphere-ai-bridge/cmd/uia-selector-report
|
|
## go run ./cmd/uia-selector-report
|
|
N14 selector report generated: catalog=10 matched=10 unmatched=0 ambiguous=0
|
|
## output safety scan
|
|
N14 UIA selector report verification passed.
|
|
```
|
|
|
|
- [ ] **Step 3: Commit verifier**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add scripts\verify-n14-uia-selector-report.ps1
|
|
git commit -m "test: add N14 selector report verifier"
|
|
```
|
|
|
|
## Task 5: Final verification and boundary check
|
|
|
|
**Files:**
|
|
- No new files.
|
|
- Verify: all N14 implementation files and unchanged helper/MCP action surfaces.
|
|
|
|
- [ ] **Step 1: Run package tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
go test ./internal/uiaselector ./cmd/uia-selector-report -count=1
|
|
```
|
|
|
|
Expected: both packages PASS.
|
|
|
|
- [ ] **Step 2: Run repository tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
go test ./...
|
|
```
|
|
|
|
Expected: all packages PASS.
|
|
|
|
If dependency download fails with a transient proxy error, retry once with:
|
|
|
|
```powershell
|
|
$env:GOPROXY = "https://goproxy.cn,direct"
|
|
go test ./...
|
|
```
|
|
|
|
- [ ] **Step 3: Run N14 verifier**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-n14-uia-selector-report.ps1
|
|
```
|
|
|
|
Expected: verifier PASS and writes ignored reports under `runs/n14-selector-report/`.
|
|
|
|
- [ ] **Step 4: Check no helper/MCP action surface changed**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git diff --name-only main..HEAD -- native/ISphereWinHelper/Program.cs internal/tools/winhelper.go cmd/isphere-mcp/main.go
|
|
```
|
|
|
|
Expected: no output.
|
|
|
|
If any of those paths appear, stop and review because N14 is offline-only.
|
|
|
|
- [ ] **Step 5: Check N14 changed files**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git diff --name-only main..HEAD
|
|
```
|
|
|
|
Expected N14 implementation files are limited to:
|
|
|
|
```text
|
|
docs/superpowers/specs/2026-07-09-n14-uia-selector-report-design.md
|
|
internal/uiaselector/report.go
|
|
internal/uiaselector/report_test.go
|
|
cmd/uia-selector-report/main.go
|
|
cmd/uia-selector-report/main_test.go
|
|
scripts/verify-n14-uia-selector-report.ps1
|
|
```
|
|
|
|
Additional plan file is acceptable if this implementation plan is committed on the branch:
|
|
|
|
```text
|
|
docs/superpowers/plans/2026-07-09-n14-uia-selector-report-implementation.md
|
|
```
|
|
|
|
- [ ] **Step 6: Commit final checkpoint if needed**
|
|
|
|
If final verification changes documentation or scripts, run:
|
|
|
|
```powershell
|
|
git add <changed files>
|
|
git commit -m "docs: record N14 selector report verification"
|
|
```
|
|
|
|
If no files changed, do not create an empty commit.
|
|
|
|
## Self-review checklist
|
|
|
|
- Spec coverage: this plan covers report model, full catalog matching, JSON report, Markdown report, strict mode, CLI, verifier, and final boundary checks.
|
|
- Boundary: no helper op, no MCP tool, no live hwnd, no UI action, no visible text export.
|
|
- TDD: every Go implementation task begins with failing tests and then minimal implementation.
|
|
- Commit cadence: report model, renderers, CLI, verifier are separate commits.
|
|
- Output safety: tests and verifier scan for exact forbidden visible-text fields and known root visible text.
|
|
- Environment: Go 1.23.x is required; use `GOPROXY=https://goproxy.cn,direct` only for transient dependency download failures. |