Merge PR #1: N12R/N13 internal sandbox handoff and UIA selector catalog
Merge N12R/N13 docs, validator fix, and offline UIA selector catalog.
This commit was merged in pull request #1.
This commit is contained in:
6
.gitignore
vendored
6
.gitignore
vendored
@@ -18,3 +18,9 @@ obj/
|
||||
dist/
|
||||
__pycache__/
|
||||
.venv/
|
||||
|
||||
# Local handoff prompt
|
||||
/给接手人的交接话术.md
|
||||
|
||||
# Local intake scratch directory
|
||||
/temp/
|
||||
|
||||
341
docs/n12r-internal-operator-minimal-retry-2026-07-09.md
Normal file
341
docs/n12r-internal-operator-minimal-retry-2026-07-09.md
Normal file
@@ -0,0 +1,341 @@
|
||||
# N12R 内网操作员最小重做步骤
|
||||
|
||||
日期:2026-07-09
|
||||
|
||||
适用对象:内网测试沙盒操作员。
|
||||
|
||||
目的:修正本次 N12R 返回包中 `selected-window.json` 缺失和 `dump_uia` 失败的问题,重新生成一个可被外网校验接受的 N12R 采集包。
|
||||
|
||||
## 1. 本次失败原因
|
||||
|
||||
本次返回包被外网校验拒收,核心原因是窗口句柄写错了。
|
||||
|
||||
日志中写成了:
|
||||
|
||||
```powershell
|
||||
$selectedHwnd = "<0x290E6E>"
|
||||
```
|
||||
|
||||
这是错误写法。`<HWND>` 里的尖括号只是提示格式,实际填写时必须删掉。
|
||||
|
||||
正确写法是:
|
||||
|
||||
```powershell
|
||||
$selectedHwnd = "0x290E6E"
|
||||
```
|
||||
|
||||
因为多了尖括号,后续窗口匹配失败:
|
||||
|
||||
```text
|
||||
selected hwnd not found in scan result: <0x290E6E>
|
||||
```
|
||||
|
||||
所以:
|
||||
|
||||
1. `windows/selected-window.json` 没有生成。
|
||||
2. `dump_uia` 使用了无效 hwnd。
|
||||
3. `uia/dump-uia-main.json` 返回 `WINDOW_NOT_FOUND`。
|
||||
|
||||
## 2. 重做前确认
|
||||
|
||||
在内网测试沙盒中打开 PowerShell,进入仓库根目录:
|
||||
|
||||
```powershell
|
||||
cd E:\isphere-ai-bridge
|
||||
```
|
||||
|
||||
确认 iSphere / IMPlatformClient 已经人工登录,并且主窗口可见、没有最小化。
|
||||
|
||||
不要执行以下动作:
|
||||
|
||||
```text
|
||||
搜索联系人
|
||||
打开会话
|
||||
发送消息
|
||||
发送文件
|
||||
接收/下载文件
|
||||
自动登录
|
||||
注入
|
||||
hook
|
||||
读取内存
|
||||
绕过安防
|
||||
```
|
||||
|
||||
## 3. 推荐做法:重新运行 scan_windows
|
||||
|
||||
窗口句柄可能已经变化,所以推荐先重新扫描窗口。
|
||||
|
||||
如果沿用原采集编号:
|
||||
|
||||
```powershell
|
||||
$captureId = "2026-07-08-internal-sandbox-a"
|
||||
$captureRoot = "runs\internal-sandbox-live-capture\$captureId"
|
||||
$helper = Resolve-Path -LiteralPath "runs\win-helper\ISphereWinHelper.exe"
|
||||
```
|
||||
|
||||
重新扫描可见窗口:
|
||||
|
||||
```powershell
|
||||
@{
|
||||
protocol = "isphere.helper.v1"
|
||||
request_id = "n12r-scan-windows-retry"
|
||||
op = "scan_windows"
|
||||
timeout_ms = 5000
|
||||
args = @{ include_all_visible = $true }
|
||||
} | ConvertTo-Json -Depth 8 -Compress |
|
||||
& $helper --json |
|
||||
Set-Content -Path "$captureRoot\windows\scan-windows.json" -Encoding UTF8
|
||||
```
|
||||
|
||||
查看候选窗口:
|
||||
|
||||
```powershell
|
||||
Get-Content -Path "$captureRoot\windows\scan-windows.json" -Encoding UTF8 | ConvertFrom-Json |
|
||||
Select-Object -ExpandProperty data |
|
||||
Select-Object -ExpandProperty windows |
|
||||
Where-Object { $_.process_name -match "iSphere|IMPlatform|IMPP" -or $_.title -match "iSphere|IMPlatform|IMPP" } |
|
||||
Format-Table hwnd,pid,process_name,title,class_name,visible,score -AutoSize
|
||||
```
|
||||
|
||||
从输出中选择 iSphere / IMPlatformClient 主窗口的 `hwnd`。
|
||||
|
||||
示例中之前的候选值是:
|
||||
|
||||
```text
|
||||
0x290E6E
|
||||
```
|
||||
|
||||
如果这次扫描输出变了,请使用新的 `hwnd`。
|
||||
|
||||
## 4. 正确保存 selected-window.json
|
||||
|
||||
把下面命令中的 `0x290E6E` 改成你刚刚确认的真实窗口句柄。
|
||||
|
||||
注意:不要加尖括号。
|
||||
|
||||
正确:
|
||||
|
||||
```powershell
|
||||
$selectedHwnd = "0x290E6E"
|
||||
```
|
||||
|
||||
错误:
|
||||
|
||||
```powershell
|
||||
$selectedHwnd = "<0x290E6E>"
|
||||
```
|
||||
|
||||
保存所选窗口元数据:
|
||||
|
||||
```powershell
|
||||
$selectedHwnd = "0x290E6E"
|
||||
|
||||
$scanObj = Get-Content -Path "$captureRoot\windows\scan-windows.json" -Encoding UTF8 | ConvertFrom-Json
|
||||
$selected = $scanObj.data.windows | Where-Object { $_.hwnd -eq $selectedHwnd } | Select-Object -First 1
|
||||
if (-not $selected) { throw "selected hwnd not found in scan result: $selectedHwnd" }
|
||||
$selected | ConvertTo-Json -Depth 8 | Set-Content -Path "$captureRoot\windows\selected-window.json" -Encoding UTF8
|
||||
```
|
||||
|
||||
确认文件已经生成:
|
||||
|
||||
```powershell
|
||||
Test-Path -LiteralPath "$captureRoot\windows\selected-window.json"
|
||||
```
|
||||
|
||||
期望输出:
|
||||
|
||||
```text
|
||||
True
|
||||
```
|
||||
|
||||
## 5. 重新执行 dump_uia
|
||||
|
||||
```powershell
|
||||
@{
|
||||
protocol = "isphere.helper.v1"
|
||||
request_id = "n12r-dump-uia-main-retry"
|
||||
op = "dump_uia"
|
||||
timeout_ms = 15000
|
||||
args = @{
|
||||
hwnd = $selectedHwnd
|
||||
max_depth = 8
|
||||
include_text = $true
|
||||
max_children = 100
|
||||
}
|
||||
} | ConvertTo-Json -Depth 12 -Compress |
|
||||
& $helper --json |
|
||||
Set-Content -Path "$captureRoot\uia\dump-uia-main.json" -Encoding UTF8
|
||||
```
|
||||
|
||||
检查结果是否成功:
|
||||
|
||||
```powershell
|
||||
$dump = Get-Content "$captureRoot\uia\dump-uia-main.json" -Encoding UTF8 | ConvertFrom-Json
|
||||
$dump.ok
|
||||
$dump.error
|
||||
```
|
||||
|
||||
期望:
|
||||
|
||||
```text
|
||||
True
|
||||
```
|
||||
|
||||
并且 `$dump.error` 应为空或为 `null`。
|
||||
|
||||
如果仍然是 `WINDOW_NOT_FOUND`,说明窗口句柄已经失效。请回到第 3 步重新扫描并选择新的 hwnd。
|
||||
|
||||
## 6. 重新创建脱敏副本
|
||||
|
||||
```powershell
|
||||
Copy-Item -LiteralPath "$captureRoot\uia\dump-uia-main.json" -Destination "$captureRoot\uia\dump-uia-main-redacted.json" -Force
|
||||
```
|
||||
|
||||
然后人工打开:
|
||||
|
||||
```text
|
||||
$captureRoot\uia\dump-uia-main-redacted.json
|
||||
```
|
||||
|
||||
脱敏可见敏感文本,例如:
|
||||
|
||||
```text
|
||||
人员姓名
|
||||
手机号
|
||||
邮箱
|
||||
消息文本
|
||||
联系人名称
|
||||
敏感业务标题
|
||||
类似 token / 密码 / cookie 的字符串
|
||||
```
|
||||
|
||||
尽量保留这些结构字段:
|
||||
|
||||
```text
|
||||
control_type
|
||||
automation_id
|
||||
class_name
|
||||
framework_id
|
||||
bounds
|
||||
is_enabled
|
||||
is_offscreen
|
||||
children
|
||||
```
|
||||
|
||||
## 7. 修正 source-notes.txt
|
||||
|
||||
打开:
|
||||
|
||||
```text
|
||||
$captureRoot\metadata\source-notes.txt
|
||||
```
|
||||
|
||||
把这些字段从 `是/否` 改成真实值:
|
||||
|
||||
```text
|
||||
iSphere manually logged in: 是
|
||||
client opened before capture: 是
|
||||
main window visible: 是
|
||||
```
|
||||
|
||||
如果真实情况不是“是”,不要继续打包,先重新完成对应动作。
|
||||
|
||||
## 8. 修正 operator-notes.txt
|
||||
|
||||
打开:
|
||||
|
||||
```text
|
||||
$captureRoot\notes\operator-notes.txt
|
||||
```
|
||||
|
||||
不要保留 `<...>` 示例内容。请改成真实说明。
|
||||
|
||||
示例:
|
||||
|
||||
```text
|
||||
打开的客户端: iSphere / IMPlatformClient
|
||||
登录方式: 人工正常公司流程
|
||||
是否使用 MFA/SSO: 是,不包含任何秘密
|
||||
所选窗口 hwnd: 0x290E6E
|
||||
选择该窗口的原因: process_name 为 IMPlatformClient,标题为国网甘肃省电力公司,visible=True,score=65
|
||||
已做脱敏: 已检查 dump-uia-main-redacted.json,移除可见敏感文本
|
||||
采集问题: 首次把 hwnd 写成 <0x290E6E> 导致失败,本次已修正为 0x290E6E 后重新采集
|
||||
forbidden actions performed: no
|
||||
```
|
||||
|
||||
## 9. 复制出去前必须验证
|
||||
|
||||
运行必需文件检查:
|
||||
|
||||
```powershell
|
||||
$required = @(
|
||||
"$captureRoot\metadata\source-notes.txt",
|
||||
"$captureRoot\helper\version.json",
|
||||
"$captureRoot\helper\self-check.json",
|
||||
"$captureRoot\windows\scan-windows.json",
|
||||
"$captureRoot\windows\selected-window.json",
|
||||
"$captureRoot\uia\dump-uia-main.json",
|
||||
"$captureRoot\uia\dump-uia-main-redacted.json",
|
||||
"$captureRoot\notes\operator-notes.txt"
|
||||
)
|
||||
foreach ($path in $required) {
|
||||
if (-not (Test-Path -LiteralPath $path)) { throw "missing required file: $path" }
|
||||
}
|
||||
"N12R retry package files present: $captureRoot" | Tee-Object -FilePath "$captureRoot\metadata\verification-summary.txt"
|
||||
```
|
||||
|
||||
运行 JSON 解析检查:
|
||||
|
||||
```powershell
|
||||
Get-Content "$captureRoot\helper\version.json" -Encoding UTF8 | ConvertFrom-Json | Out-Null
|
||||
Get-Content "$captureRoot\helper\self-check.json" -Encoding UTF8 | ConvertFrom-Json | Out-Null
|
||||
Get-Content "$captureRoot\windows\scan-windows.json" -Encoding UTF8 | ConvertFrom-Json | Out-Null
|
||||
Get-Content "$captureRoot\windows\selected-window.json" -Encoding UTF8 | ConvertFrom-Json | Out-Null
|
||||
Get-Content "$captureRoot\uia\dump-uia-main.json" -Encoding UTF8 | ConvertFrom-Json | Out-Null
|
||||
Get-Content "$captureRoot\uia\dump-uia-main-redacted.json" -Encoding UTF8 | ConvertFrom-Json | Out-Null
|
||||
```
|
||||
|
||||
检查 UIA dump 必须是成功:
|
||||
|
||||
```powershell
|
||||
$dump = Get-Content "$captureRoot\uia\dump-uia-main.json" -Encoding UTF8 | ConvertFrom-Json
|
||||
if (-not $dump.ok) { throw "dump_uia failed: $($dump.error | ConvertTo-Json -Depth 8 -Compress)" }
|
||||
```
|
||||
|
||||
## 10. 重新压缩返回
|
||||
|
||||
```powershell
|
||||
Compress-Archive -LiteralPath $captureRoot -DestinationPath "$captureRoot.zip" -Force
|
||||
```
|
||||
|
||||
把新的 zip 或目录返回给外网协调环境。
|
||||
|
||||
返回时请同时说明:
|
||||
|
||||
```text
|
||||
N12R 重采/补采包已准备好。
|
||||
路径: runs/internal-sandbox-live-capture/2026-07-08-internal-sandbox-a/
|
||||
人工 iSphere 登录: 是
|
||||
采集期间主窗口可见: 是
|
||||
forbidden actions performed: no
|
||||
本次已修正 hwnd,不再包含尖括号;selected-window.json 已生成;dump_uia ok=true
|
||||
```
|
||||
|
||||
## 11. 外网收到后如何判断
|
||||
|
||||
外网协调环境会重新运行:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\validate-n12r-return-package.ps1 `
|
||||
-PackageRoot <返回包目录> `
|
||||
-ReportPath <包外报告路径>
|
||||
```
|
||||
|
||||
只有 validator 输出:
|
||||
|
||||
```text
|
||||
Decision: ACCEPT
|
||||
```
|
||||
|
||||
才能进入 N13 selector 设计。
|
||||
|
||||
@@ -0,0 +1,943 @@
|
||||
# N13 UIA Selector Catalog Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build an offline, read-only N13 UIA selector catalog and matcher that can validate selectors against the accepted N12R UIA dump without adding live helper or MCP actions.
|
||||
|
||||
**Architecture:** Add a focused Go package `internal/uiaselector` with selector data types, a static first-pass catalog, UIA dump loading, and pure in-memory matching. Tests use a redacted N12R fixture and assert that matching returns structural node summaries without visible text.
|
||||
|
||||
**Tech Stack:** Go 1.23.x, standard library `encoding/json`, existing repository test style, PowerShell verification wrapper.
|
||||
|
||||
---
|
||||
|
||||
## Scope and boundary
|
||||
|
||||
This plan implements only offline selector catalog and matching.
|
||||
|
||||
It does not modify these files:
|
||||
|
||||
```text
|
||||
native/ISphereWinHelper/Program.cs
|
||||
internal/tools/winhelper.go
|
||||
cmd/isphere-mcp/main.go
|
||||
```
|
||||
|
||||
It does not add MCP tools or helper ops. It does not click, type, open conversations, send messages, send files, receive files, or read chat content.
|
||||
|
||||
The accepted N12R evidence used by tests is:
|
||||
|
||||
```text
|
||||
runs/internal-sandbox-live-capture-intake/2026-07-09-retry-2/2026-07-08-internal-sandbox-a/uia/dump-uia-main-redacted.json
|
||||
```
|
||||
|
||||
## File structure
|
||||
|
||||
Create these files:
|
||||
|
||||
```text
|
||||
internal/uiaselector/catalog.go
|
||||
internal/uiaselector/catalog_test.go
|
||||
internal/uiaselector/dump.go
|
||||
internal/uiaselector/dump_test.go
|
||||
internal/uiaselector/matcher.go
|
||||
internal/uiaselector/matcher_test.go
|
||||
internal/uiaselector/testdata/n12r-2026-07-09-uia-redacted.json
|
||||
scripts/verify-n13-uia-selectors.ps1
|
||||
```
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- `catalog.go`: selector data structures and initial N13 selector catalog.
|
||||
- `catalog_test.go`: catalog completeness and boundary tests.
|
||||
- `dump.go`: UIA dump response and node loading helpers.
|
||||
- `dump_test.go`: fixture parse tests.
|
||||
- `matcher.go`: pure in-memory selector matcher.
|
||||
- `matcher_test.go`: matching behavior and no-visible-text result tests.
|
||||
- `testdata/n12r-2026-07-09-uia-redacted.json`: redacted accepted N12R UIA fixture copied from the ignored run artifact.
|
||||
- `scripts/verify-n13-uia-selectors.ps1`: convenience verification script for Go-enabled environments.
|
||||
|
||||
## Task 1: Add N12R redacted UIA fixture
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/uiaselector/testdata/n12r-2026-07-09-uia-redacted.json`
|
||||
|
||||
- [ ] **Step 1: Create package directory and copy the accepted redacted fixture**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
New-Item -ItemType Directory -Force -Path internal\uiaselector\testdata | Out-Null
|
||||
Copy-Item `
|
||||
-LiteralPath runs\internal-sandbox-live-capture-intake\2026-07-09-retry-2\2026-07-08-internal-sandbox-a\uia\dump-uia-main-redacted.json `
|
||||
-Destination internal\uiaselector\testdata\n12r-2026-07-09-uia-redacted.json `
|
||||
-Force
|
||||
```
|
||||
|
||||
Expected: `internal\uiaselector\testdata\n12r-2026-07-09-uia-redacted.json` exists.
|
||||
|
||||
- [ ] **Step 2: Verify fixture parses as JSON**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
Get-Content internal\uiaselector\testdata\n12r-2026-07-09-uia-redacted.json -Encoding UTF8 | ConvertFrom-Json | Out-Null
|
||||
```
|
||||
|
||||
Expected: command exits 0.
|
||||
|
||||
- [ ] **Step 3: Commit fixture**
|
||||
|
||||
```powershell
|
||||
git add internal\uiaselector\testdata\n12r-2026-07-09-uia-redacted.json
|
||||
git commit -m "test: add N13 redacted UIA fixture"
|
||||
```
|
||||
|
||||
## Task 2: Define selector catalog types and initial catalog
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/uiaselector/catalog_test.go`
|
||||
- Create: `internal/uiaselector/catalog.go`
|
||||
|
||||
- [ ] **Step 1: Write failing catalog tests**
|
||||
|
||||
Create `internal/uiaselector/catalog_test.go`:
|
||||
|
||||
```go
|
||||
package uiaselector
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestDefaultCatalogContainsInitialN13Selectors(t *testing.T) {
|
||||
catalog := DefaultCatalog()
|
||||
wantIDs := []string{
|
||||
"main_window",
|
||||
"main_plugin_tab",
|
||||
"home_page_window",
|
||||
"tab_page_content",
|
||||
"left_panel",
|
||||
"roster_panel",
|
||||
"roster_view",
|
||||
"fuzzy_query_panel",
|
||||
"fuzzy_query_window",
|
||||
"fuzzy_search_edit",
|
||||
}
|
||||
|
||||
byID := map[string]Selector{}
|
||||
for _, selector := range catalog {
|
||||
byID[selector.ID] = selector
|
||||
}
|
||||
|
||||
for _, id := range wantIDs {
|
||||
selector, ok := byID[id]
|
||||
if !ok {
|
||||
t.Fatalf("DefaultCatalog missing selector %q", id)
|
||||
}
|
||||
if selector.AllowedUse != AllowedUseReadOnlyLocate {
|
||||
t.Fatalf("selector %q allowed_use = %q, want %q", id, selector.AllowedUse, AllowedUseReadOnlyLocate)
|
||||
}
|
||||
if selector.Required.AutomationID == "" {
|
||||
t.Fatalf("selector %q must require automation_id", id)
|
||||
}
|
||||
if selector.Required.ControlType == "" {
|
||||
t.Fatalf("selector %q must require control_type", id)
|
||||
}
|
||||
if selector.StabilityScore < 80 {
|
||||
t.Fatalf("selector %q stability_score = %d, want >= 80", id, selector.StabilityScore)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDefaultCatalogDoesNotDescribeActionTools(t *testing.T) {
|
||||
for _, selector := range DefaultCatalog() {
|
||||
if selector.AllowedUse != AllowedUseReadOnlyLocate {
|
||||
t.Fatalf("selector %q allowed_use = %q, want read-only", selector.ID, selector.AllowedUse)
|
||||
}
|
||||
lower := strings.ToLower(selector.ID + " " + selector.Description)
|
||||
for _, forbidden := range []string{"send", "upload", "download", "receive", "conversation", "login", "file_transfer"} {
|
||||
if strings.Contains(lower, forbidden) {
|
||||
t.Fatalf("selector %q contains action word %q", selector.ID, forbidden)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run catalog tests and verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test ./internal/uiaselector -run TestDefaultCatalog -count=1
|
||||
```
|
||||
|
||||
Expected: FAIL because package `internal/uiaselector` or `DefaultCatalog` does not exist.
|
||||
|
||||
If this environment reports `go` not found, stop execution and record the environment gap. Continue only in a Go 1.23.x environment.
|
||||
|
||||
- [ ] **Step 3: Implement catalog types and selectors**
|
||||
|
||||
Create `internal/uiaselector/catalog.go`:
|
||||
|
||||
```go
|
||||
package uiaselector
|
||||
|
||||
const AllowedUseReadOnlyLocate = "read_only_locate"
|
||||
|
||||
type Selector struct {
|
||||
ID string `json:"id"`
|
||||
Description string `json:"description"`
|
||||
AllowedUse string `json:"allowed_use"`
|
||||
Required Criteria `json:"required"`
|
||||
Preferred PreferredCriteria `json:"preferred"`
|
||||
Fallback []FallbackCriteria `json:"fallback"`
|
||||
StabilityScore int `json:"stability_score"`
|
||||
Evidence Evidence `json:"evidence"`
|
||||
}
|
||||
|
||||
type Criteria struct {
|
||||
AutomationID string `json:"automation_id,omitempty"`
|
||||
ControlType string `json:"control_type,omitempty"`
|
||||
FrameworkID string `json:"framework_id,omitempty"`
|
||||
ClassName string `json:"class_name,omitempty"`
|
||||
ClassPrefix string `json:"class_name_prefix,omitempty"`
|
||||
AncestorID string `json:"ancestor_selector,omitempty"`
|
||||
}
|
||||
|
||||
type PreferredCriteria struct {
|
||||
ClassName string `json:"class_name,omitempty"`
|
||||
ClassPrefix string `json:"class_name_prefix,omitempty"`
|
||||
PathHint string `json:"path_hint,omitempty"`
|
||||
AncestorID string `json:"ancestor_selector,omitempty"`
|
||||
}
|
||||
|
||||
type FallbackCriteria struct {
|
||||
AncestorID string `json:"ancestor_selector,omitempty"`
|
||||
ControlType string `json:"control_type,omitempty"`
|
||||
ClassPrefix string `json:"class_name_prefix,omitempty"`
|
||||
ChildAutomationIDContains []string `json:"child_automation_id_contains,omitempty"`
|
||||
ChildControlTypeContains []string `json:"child_control_type_contains,omitempty"`
|
||||
}
|
||||
|
||||
type Evidence struct {
|
||||
CaptureID string `json:"capture_id"`
|
||||
UIAPath string `json:"uia_path"`
|
||||
}
|
||||
|
||||
func DefaultCatalog() []Selector {
|
||||
return []Selector{
|
||||
{
|
||||
ID: "main_window",
|
||||
Description: "iSphere / IMPlatformClient 主窗口",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "frmMain", ControlType: "Window", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{ClassPrefix: "WindowsForms10.Window.8.app.0", PathHint: "root"},
|
||||
Fallback: []FallbackCriteria{{ControlType: "Window", ClassPrefix: "WindowsForms10.Window.8.app.0", ChildAutomationIDContains: []string{"panelLeft", "ucPluginTabControl1"}}},
|
||||
StabilityScore: 95,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root"},
|
||||
},
|
||||
{
|
||||
ID: "main_plugin_tab",
|
||||
Description: "主功能 tab 容器",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "ucPluginTabControl1", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/3", AncestorID: "main_window"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "main_window", ControlType: "Pane", ChildAutomationIDContains: []string{"FrmHomePage", "ucTabPage"}}},
|
||||
StabilityScore: 90,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/3"},
|
||||
},
|
||||
{
|
||||
ID: "home_page_window",
|
||||
Description: "首页内容窗口",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "FrmHomePage", ControlType: "Window", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/3/0/0", AncestorID: "main_plugin_tab"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "main_plugin_tab", ControlType: "Window", ClassPrefix: "WindowsForms10.Window.8.app.0"}},
|
||||
StabilityScore: 82,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/3/0/0"},
|
||||
},
|
||||
{
|
||||
ID: "tab_page_content",
|
||||
Description: "tab 页内容窗口",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "FrmTabPageContent", ControlType: "Window", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/3/1/0", AncestorID: "main_plugin_tab"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "main_plugin_tab", ControlType: "Window", ChildControlTypeContains: []string{"TitleBar"}}},
|
||||
StabilityScore: 82,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/3/1/0"},
|
||||
},
|
||||
{
|
||||
ID: "left_panel",
|
||||
Description: "左侧主面板",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "panelLeft", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4", AncestorID: "main_window"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "main_window", ControlType: "Pane", ChildAutomationIDContains: []string{"ucPnlRoster1", "ucFuzzyQuery1"}}},
|
||||
StabilityScore: 92,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4"},
|
||||
},
|
||||
{
|
||||
ID: "roster_panel",
|
||||
Description: "组织/联系人容器外层",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "ucPnlRoster1", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4/0/0", AncestorID: "left_panel"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "left_panel", ControlType: "Pane", ChildAutomationIDContains: []string{"ucRoster1"}}},
|
||||
StabilityScore: 88,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/0/0"},
|
||||
},
|
||||
{
|
||||
ID: "roster_view",
|
||||
Description: "组织/联系人列表区域",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "ucRoster1", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4/0/0/0", AncestorID: "roster_panel"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "roster_panel", ControlType: "Pane", ClassPrefix: "WindowsForms10.Window.8.app.0"}},
|
||||
StabilityScore: 86,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/0/0/0"},
|
||||
},
|
||||
{
|
||||
ID: "fuzzy_query_panel",
|
||||
Description: "模糊搜索组件外层",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "ucFuzzyQuery1", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4/1/0", AncestorID: "left_panel"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "left_panel", ControlType: "Pane", ChildAutomationIDContains: []string{"FrmFuzzyQuery"}}},
|
||||
StabilityScore: 90,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/1/0"},
|
||||
},
|
||||
{
|
||||
ID: "fuzzy_query_window",
|
||||
Description: "模糊搜索内部窗口",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "FrmFuzzyQuery", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4/1/0/0", AncestorID: "fuzzy_query_panel"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "fuzzy_query_panel", ControlType: "Pane", ChildAutomationIDContains: []string{"skinAlphaTxt"}}},
|
||||
StabilityScore: 88,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/1/0/0"},
|
||||
},
|
||||
{
|
||||
ID: "fuzzy_search_edit",
|
||||
Description: "模糊搜索输入框",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "skinAlphaTxt", ControlType: "Edit", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{ClassName: "WindowsForms10.EDIT.app.0.d3a00f_r29_ad1", PathHint: "root/4/1/0/0/2", AncestorID: "fuzzy_query_window"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "fuzzy_query_window", ControlType: "Edit", ClassPrefix: "WindowsForms10.EDIT.app.0"}},
|
||||
StabilityScore: 93,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/1/0/0/2"},
|
||||
},
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run catalog tests and verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test ./internal/uiaselector -run TestDefaultCatalog -count=1
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit catalog**
|
||||
|
||||
```powershell
|
||||
git add internal\uiaselector\catalog.go internal\uiaselector\catalog_test.go
|
||||
git commit -m "feat: add N13 UIA selector catalog"
|
||||
```
|
||||
|
||||
## Task 3: Load redacted UIA dump fixture
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/uiaselector/dump_test.go`
|
||||
- Create: `internal/uiaselector/dump.go`
|
||||
|
||||
- [ ] **Step 1: Write failing dump loading tests**
|
||||
|
||||
Create `internal/uiaselector/dump_test.go`:
|
||||
|
||||
```go
|
||||
package uiaselector
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestLoadDumpFixtureParsesAcceptedN12RTree(t *testing.T) {
|
||||
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadDumpFile: %v", err)
|
||||
}
|
||||
if root.AutomationID != "frmMain" {
|
||||
t.Fatalf("root automation_id = %q, want frmMain", root.AutomationID)
|
||||
}
|
||||
if root.ControlType != "Window" {
|
||||
t.Fatalf("root control_type = %q, want Window", root.ControlType)
|
||||
}
|
||||
if got := CountNodes(root); got != 66 {
|
||||
t.Fatalf("CountNodes = %d, want 66", got)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run dump tests and verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test ./internal/uiaselector -run TestLoadDumpFixture -count=1
|
||||
```
|
||||
|
||||
Expected: FAIL because `LoadDumpFile` and `CountNodes` do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement dump loading**
|
||||
|
||||
Create `internal/uiaselector/dump.go`:
|
||||
|
||||
```go
|
||||
package uiaselector
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
)
|
||||
|
||||
type DumpResponse struct {
|
||||
OK bool `json:"ok"`
|
||||
Data *DumpData `json:"data"`
|
||||
Error any `json:"error"`
|
||||
}
|
||||
|
||||
type DumpData struct {
|
||||
Root *Node `json:"root"`
|
||||
}
|
||||
|
||||
type Node struct {
|
||||
Name string `json:"name"`
|
||||
ControlType string `json:"control_type"`
|
||||
AutomationID string `json:"automation_id"`
|
||||
ClassName string `json:"class_name"`
|
||||
FrameworkID string `json:"framework_id"`
|
||||
IsEnabled *bool `json:"is_enabled"`
|
||||
IsOffscreen *bool `json:"is_offscreen"`
|
||||
Bounds any `json:"bounds"`
|
||||
Children []Node `json:"children"`
|
||||
}
|
||||
|
||||
func LoadDumpFile(path string) (*Node, error) {
|
||||
content, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var response DumpResponse
|
||||
if err := json.Unmarshal(content, &response); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !response.OK {
|
||||
return nil, fmt.Errorf("dump response ok=false: %v", response.Error)
|
||||
}
|
||||
if response.Data == nil || response.Data.Root == nil {
|
||||
return nil, fmt.Errorf("dump response has no data.root")
|
||||
}
|
||||
return response.Data.Root, nil
|
||||
}
|
||||
|
||||
func CountNodes(root *Node) int {
|
||||
if root == nil {
|
||||
return 0
|
||||
}
|
||||
count := 1
|
||||
for i := range root.Children {
|
||||
count += CountNodes(&root.Children[i])
|
||||
}
|
||||
return count
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run dump tests and verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test ./internal/uiaselector -run TestLoadDumpFixture -count=1
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit dump loader**
|
||||
|
||||
```powershell
|
||||
git add internal\uiaselector\dump.go internal\uiaselector\dump_test.go
|
||||
git commit -m "feat: load N13 UIA dump fixture"
|
||||
```
|
||||
|
||||
## Task 4: Implement read-only selector matcher
|
||||
|
||||
**Files:**
|
||||
- Create: `internal/uiaselector/matcher_test.go`
|
||||
- Create: `internal/uiaselector/matcher.go`
|
||||
|
||||
- [ ] **Step 1: Write failing matcher tests**
|
||||
|
||||
Create `internal/uiaselector/matcher_test.go`:
|
||||
|
||||
```go
|
||||
package uiaselector
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestMatchSelectorFindsKnownN12RNodes(t *testing.T) {
|
||||
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadDumpFile: %v", err)
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
selectorID string
|
||||
path string
|
||||
control string
|
||||
autoID string
|
||||
}{
|
||||
{selectorID: "main_window", path: "root", control: "Window", autoID: "frmMain"},
|
||||
{selectorID: "left_panel", path: "root/4", control: "Pane", autoID: "panelLeft"},
|
||||
{selectorID: "fuzzy_search_edit", path: "root/4/1/0/0/2", control: "Edit", autoID: "skinAlphaTxt"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
result := MatchSelector(root, DefaultCatalog(), tt.selectorID)
|
||||
if !result.OK {
|
||||
t.Fatalf("%s OK=false: %#v", tt.selectorID, result.Error)
|
||||
}
|
||||
if !result.Matched {
|
||||
t.Fatalf("%s matched=false", tt.selectorID)
|
||||
}
|
||||
if result.MatchCount != 1 {
|
||||
t.Fatalf("%s match_count = %d, want 1", tt.selectorID, result.MatchCount)
|
||||
}
|
||||
match := result.Matches[0]
|
||||
if match.Path != tt.path || match.ControlType != tt.control || match.AutomationID != tt.autoID {
|
||||
t.Fatalf("%s match = %#v", tt.selectorID, match)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMatchSelectorDoesNotReturnVisibleText(t *testing.T) {
|
||||
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadDumpFile: %v", err)
|
||||
}
|
||||
result := MatchSelector(root, DefaultCatalog(), "main_window")
|
||||
if !result.OK || !result.Matched || len(result.Matches) != 1 {
|
||||
t.Fatalf("unexpected result: %#v", result)
|
||||
}
|
||||
match := result.Matches[0]
|
||||
if match.Name != "" {
|
||||
t.Fatalf("match returned visible name text %q", match.Name)
|
||||
}
|
||||
if !match.HasName {
|
||||
t.Fatalf("main_window should report has_name=true")
|
||||
}
|
||||
if match.NameLength == 0 {
|
||||
t.Fatalf("main_window should report non-zero name_length")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMatchSelectorUnknownSelectorIsStructuredFailure(t *testing.T) {
|
||||
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadDumpFile: %v", err)
|
||||
}
|
||||
result := MatchSelector(root, DefaultCatalog(), "missing_selector")
|
||||
if result.OK {
|
||||
t.Fatalf("OK=true for missing selector: %#v", result)
|
||||
}
|
||||
if result.Error.Code != "UNKNOWN_SELECTOR" {
|
||||
t.Fatalf("error code = %q, want UNKNOWN_SELECTOR", result.Error.Code)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run matcher tests and verify they fail**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test ./internal/uiaselector -run TestMatchSelector -count=1
|
||||
```
|
||||
|
||||
Expected: FAIL because `MatchSelector` and result types do not exist.
|
||||
|
||||
- [ ] **Step 3: Implement matcher**
|
||||
|
||||
Create `internal/uiaselector/matcher.go`:
|
||||
|
||||
```go
|
||||
package uiaselector
|
||||
|
||||
import "strings"
|
||||
|
||||
type MatchResult struct {
|
||||
OK bool `json:"ok"`
|
||||
SelectorID string `json:"selector_id"`
|
||||
Matched bool `json:"matched"`
|
||||
MatchCount int `json:"match_count"`
|
||||
Matches []NodeMatch `json:"matches"`
|
||||
Warnings []string `json:"warnings"`
|
||||
Error MatchError `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
type MatchError struct {
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
type NodeMatch struct {
|
||||
Path string `json:"path"`
|
||||
ControlType string `json:"control_type"`
|
||||
AutomationID string `json:"automation_id"`
|
||||
ClassName string `json:"class_name"`
|
||||
FrameworkID string `json:"framework_id"`
|
||||
HasName bool `json:"has_name"`
|
||||
NameLength int `json:"name_length"`
|
||||
Name string `json:"-"`
|
||||
}
|
||||
|
||||
type nodeRef struct {
|
||||
path string
|
||||
node *Node
|
||||
}
|
||||
|
||||
func MatchSelector(root *Node, catalog []Selector, selectorID string) MatchResult {
|
||||
selector, ok := findSelector(catalog, selectorID)
|
||||
if !ok {
|
||||
return MatchResult{OK: false, SelectorID: selectorID, Error: MatchError{Code: "UNKNOWN_SELECTOR", Message: "selector is not in catalog"}}
|
||||
}
|
||||
if selector.AllowedUse != AllowedUseReadOnlyLocate {
|
||||
return MatchResult{OK: false, SelectorID: selectorID, Error: MatchError{Code: "INVALID_SELECTOR", Message: "selector is not read-only"}}
|
||||
}
|
||||
refs := flatten(root)
|
||||
matches := matchCriteria(refs, selector.Required)
|
||||
if len(matches) == 0 {
|
||||
for _, fallback := range selector.Fallback {
|
||||
matches = matchFallback(refs, fallback)
|
||||
if len(matches) > 0 {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
result := MatchResult{OK: true, SelectorID: selectorID, Matched: len(matches) > 0, MatchCount: len(matches), Warnings: []string{}}
|
||||
for _, ref := range matches {
|
||||
result.Matches = append(result.Matches, summarizeNode(ref))
|
||||
}
|
||||
if !result.Matched {
|
||||
result.Warnings = append(result.Warnings, "selector_not_found")
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func findSelector(catalog []Selector, selectorID string) (Selector, bool) {
|
||||
for _, selector := range catalog {
|
||||
if selector.ID == selectorID {
|
||||
return selector, true
|
||||
}
|
||||
}
|
||||
return Selector{}, false
|
||||
}
|
||||
|
||||
func flatten(root *Node) []nodeRef {
|
||||
refs := []nodeRef{}
|
||||
var walk func(node *Node, path string)
|
||||
walk = func(node *Node, path string) {
|
||||
if node == nil {
|
||||
return
|
||||
}
|
||||
refs = append(refs, nodeRef{path: path, node: node})
|
||||
for i := range node.Children {
|
||||
walk(&node.Children[i], path+"/"+itoa(i))
|
||||
}
|
||||
}
|
||||
walk(root, "root")
|
||||
return refs
|
||||
}
|
||||
|
||||
func itoa(value int) string {
|
||||
if value == 0 {
|
||||
return "0"
|
||||
}
|
||||
digits := []byte{}
|
||||
for value > 0 {
|
||||
digits = append([]byte{byte('0' + value%10)}, digits...)
|
||||
value = value / 10
|
||||
}
|
||||
return string(digits)
|
||||
}
|
||||
|
||||
func matchCriteria(refs []nodeRef, criteria Criteria) []nodeRef {
|
||||
matches := []nodeRef{}
|
||||
for _, ref := range refs {
|
||||
if criteria.AutomationID != "" && ref.node.AutomationID != criteria.AutomationID {
|
||||
continue
|
||||
}
|
||||
if criteria.ControlType != "" && ref.node.ControlType != criteria.ControlType {
|
||||
continue
|
||||
}
|
||||
if criteria.FrameworkID != "" && ref.node.FrameworkID != criteria.FrameworkID {
|
||||
continue
|
||||
}
|
||||
if criteria.ClassName != "" && ref.node.ClassName != criteria.ClassName {
|
||||
continue
|
||||
}
|
||||
if criteria.ClassPrefix != "" && !strings.HasPrefix(ref.node.ClassName, criteria.ClassPrefix) {
|
||||
continue
|
||||
}
|
||||
matches = append(matches, ref)
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
||||
func matchFallback(refs []nodeRef, fallback FallbackCriteria) []nodeRef {
|
||||
matches := []nodeRef{}
|
||||
for _, ref := range refs {
|
||||
if fallback.ControlType != "" && ref.node.ControlType != fallback.ControlType {
|
||||
continue
|
||||
}
|
||||
if fallback.ClassPrefix != "" && !strings.HasPrefix(ref.node.ClassName, fallback.ClassPrefix) {
|
||||
continue
|
||||
}
|
||||
if !hasChildAutomationIDs(ref.node, fallback.ChildAutomationIDContains) {
|
||||
continue
|
||||
}
|
||||
if !hasChildControlTypes(ref.node, fallback.ChildControlTypeContains) {
|
||||
continue
|
||||
}
|
||||
matches = append(matches, ref)
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
||||
func hasChildAutomationIDs(node *Node, ids []string) bool {
|
||||
for _, id := range ids {
|
||||
found := false
|
||||
for i := range node.Children {
|
||||
if node.Children[i].AutomationID == id {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func hasChildControlTypes(node *Node, controlTypes []string) bool {
|
||||
for _, controlType := range controlTypes {
|
||||
found := false
|
||||
for i := range node.Children {
|
||||
if node.Children[i].ControlType == controlType {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func summarizeNode(ref nodeRef) NodeMatch {
|
||||
return NodeMatch{
|
||||
Path: ref.path,
|
||||
ControlType: ref.node.ControlType,
|
||||
AutomationID: ref.node.AutomationID,
|
||||
ClassName: ref.node.ClassName,
|
||||
FrameworkID: ref.node.FrameworkID,
|
||||
HasName: ref.node.Name != "",
|
||||
NameLength: len(ref.node.Name),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run matcher tests and verify they pass**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test ./internal/uiaselector -run TestMatchSelector -count=1
|
||||
```
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 5: Commit matcher**
|
||||
|
||||
```powershell
|
||||
git add internal\uiaselector\matcher.go internal\uiaselector\matcher_test.go
|
||||
git commit -m "feat: match N13 UIA selectors offline"
|
||||
```
|
||||
|
||||
## Task 5: Add verification script
|
||||
|
||||
**Files:**
|
||||
- Create: `scripts/verify-n13-uia-selectors.ps1`
|
||||
|
||||
- [ ] **Step 1: Create verification script**
|
||||
|
||||
Create `scripts/verify-n13-uia-selectors.ps1`:
|
||||
|
||||
```powershell
|
||||
$ErrorActionPreference = "Stop"
|
||||
Set-StrictMode -Version Latest
|
||||
|
||||
$repo = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path
|
||||
Set-Location -LiteralPath $repo
|
||||
|
||||
if (-not (Get-Command go -ErrorAction SilentlyContinue)) {
|
||||
throw "go was not found. Run this verifier in a Go 1.23.x environment."
|
||||
}
|
||||
|
||||
Write-Host "## go test ./internal/uiaselector"
|
||||
go test ./internal/uiaselector -count=1
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "go test ./internal/uiaselector failed with exit code $LASTEXITCODE"
|
||||
}
|
||||
|
||||
Write-Host "## boundary grep"
|
||||
$forbidden = @(
|
||||
"send_after_approval",
|
||||
"send_file_after_approval",
|
||||
"receive_file_after_approval",
|
||||
"open_conversation",
|
||||
"ValuePattern.SetValue",
|
||||
"InvokePattern"
|
||||
)
|
||||
$files = @(
|
||||
"internal\uiaselector\catalog.go",
|
||||
"internal\uiaselector\matcher.go"
|
||||
)
|
||||
foreach ($term in $forbidden) {
|
||||
$hit = Select-String -LiteralPath $files -Pattern $term -SimpleMatch -ErrorAction SilentlyContinue
|
||||
if ($hit) {
|
||||
throw "forbidden action term found in selector implementation: $term"
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host "N13 UIA selector verification passed."
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run script and verify it passes**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-n13-uia-selectors.ps1
|
||||
```
|
||||
|
||||
Expected in Go-enabled environment:
|
||||
|
||||
```text
|
||||
## go test ./internal/uiaselector
|
||||
ok isphere-ai-bridge/internal/uiaselector
|
||||
## boundary grep
|
||||
N13 UIA selector verification passed.
|
||||
```
|
||||
|
||||
If `go` is unavailable, expected failure is:
|
||||
|
||||
```text
|
||||
go was not found. Run this verifier in a Go 1.23.x environment.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit verification script**
|
||||
|
||||
```powershell
|
||||
git add scripts\verify-n13-uia-selectors.ps1
|
||||
git commit -m "test: add N13 selector verifier"
|
||||
```
|
||||
|
||||
## Task 6: Final verification and checkpoint
|
||||
|
||||
**Files:**
|
||||
- No new files.
|
||||
- Verify: `internal/uiaselector/*`, `scripts/verify-n13-uia-selectors.ps1`
|
||||
|
||||
- [ ] **Step 1: Run package tests**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test ./internal/uiaselector -count=1
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
ok isphere-ai-bridge/internal/uiaselector
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run repository tests if Go is available**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
go test ./...
|
||||
```
|
||||
|
||||
Expected: PASS for all packages.
|
||||
|
||||
If the environment lacks Go, do not mark implementation complete. Move to a Go 1.23.x environment and rerun.
|
||||
|
||||
- [ ] **Step 3: Run N13 verifier**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-n13-uia-selectors.ps1
|
||||
```
|
||||
|
||||
Expected: PASS in Go-enabled environment.
|
||||
|
||||
- [ ] **Step 4: Check no MCP/helper action surface changed**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git diff --name-only origin/main..HEAD
|
||||
```
|
||||
|
||||
Expected changed implementation files are limited to:
|
||||
|
||||
```text
|
||||
internal/uiaselector/catalog.go
|
||||
internal/uiaselector/catalog_test.go
|
||||
internal/uiaselector/dump.go
|
||||
internal/uiaselector/dump_test.go
|
||||
internal/uiaselector/matcher.go
|
||||
internal/uiaselector/matcher_test.go
|
||||
internal/uiaselector/testdata/n12r-2026-07-09-uia-redacted.json
|
||||
scripts/verify-n13-uia-selectors.ps1
|
||||
```
|
||||
|
||||
If `native/ISphereWinHelper/Program.cs`, `internal/tools/winhelper.go`, or `cmd/isphere-mcp/main.go` appears, stop and review because N13 is not supposed to add live tools.
|
||||
|
||||
- [ ] **Step 5: Commit final checkpoint if needed**
|
||||
|
||||
If any verification-only documentation changes were made, run:
|
||||
|
||||
```powershell
|
||||
git add <changed verification docs>
|
||||
git commit -m "docs: record N13 selector verification"
|
||||
```
|
||||
|
||||
If no files changed, do not create an empty commit.
|
||||
|
||||
## Self-review checklist
|
||||
|
||||
- Spec coverage: this plan covers selector catalog, fixture loading, read-only matcher, no visible text in result, and boundary verification.
|
||||
- Boundary: no helper op, no MCP tool, no click/input/send/file/conversation behavior.
|
||||
- TDD: every implementation task starts with failing tests before code.
|
||||
- Commit cadence: fixture, catalog, loader, matcher, verifier are separate commits.
|
||||
- Environment caveat: Go 1.23.x is required for implementation verification.
|
||||
@@ -0,0 +1,597 @@
|
||||
# N13 UIA Selector Catalog Design
|
||||
|
||||
日期:2026-07-09
|
||||
|
||||
## 1. 目标
|
||||
|
||||
N13 的目标是基于已 ACCEPT 的 N12R live UIA 证据,设计一套只读 selector catalog 和定位验证流程。
|
||||
|
||||
本阶段只做设计,不新增 helper op,不新增 MCP tool,不点击、不输入、不打开会话、不发送消息、不传输文件。
|
||||
|
||||
N13 完成后,下一步才可以写独立实现计划,考虑是否新增只读定位验证能力。
|
||||
|
||||
## 2. 输入证据
|
||||
|
||||
N12R 返回包已通过外网 validator:
|
||||
|
||||
```text
|
||||
runs/internal-sandbox-live-capture-reports/2026-07-09-retry-2/2026-07-08-internal-sandbox-a-validation.md
|
||||
Decision: ACCEPT
|
||||
PASS: 36
|
||||
WARN: 1
|
||||
FAIL: 0
|
||||
```
|
||||
|
||||
使用的 UIA dump:
|
||||
|
||||
```text
|
||||
runs/internal-sandbox-live-capture-intake/2026-07-09-retry-2/2026-07-08-internal-sandbox-a/uia/dump-uia-main.json
|
||||
```
|
||||
|
||||
关键事实:
|
||||
|
||||
```text
|
||||
root control_type: Window
|
||||
root automation_id: frmMain
|
||||
root class_name: WindowsForms10.Window.8.app.0.d3a00f_r29_ad1
|
||||
framework_id: WinForm
|
||||
UIA node count: 66
|
||||
max_depth: 7
|
||||
```
|
||||
|
||||
本设计只引用结构性字段,不依赖敏感可见文本。
|
||||
|
||||
## 3. 非目标
|
||||
|
||||
N13 不做以下事项:
|
||||
|
||||
```text
|
||||
自动登录
|
||||
搜索联系人
|
||||
打开会话
|
||||
点击控件
|
||||
输入文本
|
||||
发送消息
|
||||
发送文件
|
||||
接收文件
|
||||
读取或导出聊天内容
|
||||
注入进程
|
||||
hook
|
||||
读取目标进程内存
|
||||
绕过终端安全/安防
|
||||
```
|
||||
|
||||
N13 也不把 `search_contacts`、`open_conversation`、`read_latest_messages` 作为当前实现目标。这些只能在 selector catalog 稳定、只读定位验证通过、并形成新的审批设计后再讨论。
|
||||
|
||||
## 4. 推荐方案
|
||||
|
||||
采用“selector catalog + 只读定位验证”方案。
|
||||
|
||||
核心思想:
|
||||
|
||||
1. 用 N12R UIA dump 建立一份 selector catalog。
|
||||
2. 每个 selector 都只描述如何识别一个 UIA 节点。
|
||||
3. selector 的初始用途只允许 read-only locate。
|
||||
4. 后续如果实现 `locate_uia`,也只能返回匹配节点摘要,不执行任何 UI 动作。
|
||||
5. 任何真实动作类能力必须另写设计、计划和审批边界。
|
||||
|
||||
## 5. Selector 数据模型
|
||||
|
||||
建议 selector 条目使用以下字段:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "main_window",
|
||||
"description": "主窗口",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "frmMain",
|
||||
"control_type": "Window",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"class_name_prefix": "WindowsForms10.Window.8.app.0",
|
||||
"path_hint": "root"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"control_type": "Window",
|
||||
"class_name_prefix": "WindowsForms10.Window.8.app.0",
|
||||
"child_automation_id_contains": ["panelLeft", "ucPluginTabControl1"]
|
||||
}
|
||||
],
|
||||
"stability_score": 95,
|
||||
"evidence": {
|
||||
"capture_id": "2026-07-08-internal-sandbox-a",
|
||||
"uia_path": "root"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
字段含义:
|
||||
|
||||
- `id`:稳定、短小、英文蛇形命名。
|
||||
- `description`:中文说明。
|
||||
- `allowed_use`:N13 固定为 `read_only_locate`。
|
||||
- `required`:必须同时满足的字段。
|
||||
- `preferred`:加分项,用于减少误匹配。
|
||||
- `fallback`:主 selector 失效时的只读匹配策略。
|
||||
- `stability_score`:0 到 100,越高越稳定。
|
||||
- `evidence`:来源证据路径和 UIA 路径。
|
||||
|
||||
## 6. 初始 selector catalog
|
||||
|
||||
### 6.1 主窗口:`main_window`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "main_window",
|
||||
"description": "iSphere / IMPlatformClient 主窗口",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "frmMain",
|
||||
"control_type": "Window",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"class_name_prefix": "WindowsForms10.Window.8.app.0",
|
||||
"path_hint": "root"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"control_type": "Window",
|
||||
"class_name_prefix": "WindowsForms10.Window.8.app.0",
|
||||
"child_automation_id_contains": ["panelLeft", "ucPluginTabControl1"]
|
||||
}
|
||||
],
|
||||
"stability_score": 95,
|
||||
"evidence": {
|
||||
"uia_path": "root"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:`automation_id=frmMain` 出现在根节点;root 是 N12R 已确认主窗口,稳定性最高。
|
||||
|
||||
### 6.2 主 tab 容器:`main_plugin_tab`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "main_plugin_tab",
|
||||
"description": "主功能 tab 容器",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "ucPluginTabControl1",
|
||||
"control_type": "Pane",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"path_hint": "root/3",
|
||||
"ancestor_selector": "main_window"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "main_window",
|
||||
"control_type": "Pane",
|
||||
"child_automation_id_contains": ["FrmHomePage", "ucTabPage"]
|
||||
}
|
||||
],
|
||||
"stability_score": 90,
|
||||
"evidence": {
|
||||
"uia_path": "root/3"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:该节点直接挂在主窗口下,automation_id 明确,子结构包含首页和 tab 内容区。
|
||||
|
||||
### 6.3 首页窗口:`home_page_window`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "home_page_window",
|
||||
"description": "首页内容窗口",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "FrmHomePage",
|
||||
"control_type": "Window",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"path_hint": "root/3/0/0",
|
||||
"ancestor_selector": "main_plugin_tab"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "main_plugin_tab",
|
||||
"control_type": "Window",
|
||||
"class_name_prefix": "WindowsForms10.Window.8.app.0"
|
||||
}
|
||||
],
|
||||
"stability_score": 82,
|
||||
"evidence": {
|
||||
"uia_path": "root/3/0/0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:该节点可用于确认当前主 UI 内容区存在,但不用于动作。
|
||||
|
||||
### 6.4 tab 内容窗口:`tab_page_content`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "tab_page_content",
|
||||
"description": "tab 页内容窗口",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "FrmTabPageContent",
|
||||
"control_type": "Window",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"path_hint": "root/3/1/0",
|
||||
"ancestor_selector": "main_plugin_tab"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "main_plugin_tab",
|
||||
"control_type": "Window",
|
||||
"child_control_type_contains": ["TitleBar"]
|
||||
}
|
||||
],
|
||||
"stability_score": 82,
|
||||
"evidence": {
|
||||
"uia_path": "root/3/1/0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:可作为后续只读结构检查的内容区域锚点。
|
||||
|
||||
### 6.5 左侧面板:`left_panel`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "left_panel",
|
||||
"description": "左侧主面板",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "panelLeft",
|
||||
"control_type": "Pane",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"path_hint": "root/4",
|
||||
"ancestor_selector": "main_window"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "main_window",
|
||||
"control_type": "Pane",
|
||||
"child_automation_id_contains": ["ucPnlRoster1", "ucFuzzyQuery1"]
|
||||
}
|
||||
],
|
||||
"stability_score": 92,
|
||||
"evidence": {
|
||||
"uia_path": "root/4"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:该节点是联系人/搜索相关区域的稳定父节点。
|
||||
|
||||
### 6.6 组织/联系人容器:`roster_panel`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "roster_panel",
|
||||
"description": "组织/联系人容器外层",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "ucPnlRoster1",
|
||||
"control_type": "Pane",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"path_hint": "root/4/0/0",
|
||||
"ancestor_selector": "left_panel"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "left_panel",
|
||||
"control_type": "Pane",
|
||||
"child_automation_id_contains": ["ucRoster1"]
|
||||
}
|
||||
],
|
||||
"stability_score": 88,
|
||||
"evidence": {
|
||||
"uia_path": "root/4/0/0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:该节点是 roster 结构外层,可以只读验证联系人区域是否存在。
|
||||
|
||||
### 6.7 组织/联系人列表:`roster_view`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "roster_view",
|
||||
"description": "组织/联系人列表区域",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "ucRoster1",
|
||||
"control_type": "Pane",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"path_hint": "root/4/0/0/0",
|
||||
"ancestor_selector": "roster_panel"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "roster_panel",
|
||||
"control_type": "Pane",
|
||||
"class_name_prefix": "WindowsForms10.Window.8.app.0"
|
||||
}
|
||||
],
|
||||
"stability_score": 86,
|
||||
"evidence": {
|
||||
"uia_path": "root/4/0/0/0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:该节点本次 child_count 为 0,不能证明列表项结构;只能作为区域存在性 selector。
|
||||
|
||||
### 6.8 搜索组件外层:`fuzzy_query_panel`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "fuzzy_query_panel",
|
||||
"description": "模糊搜索组件外层",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "ucFuzzyQuery1",
|
||||
"control_type": "Pane",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"path_hint": "root/4/1/0",
|
||||
"ancestor_selector": "left_panel"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "left_panel",
|
||||
"control_type": "Pane",
|
||||
"child_automation_id_contains": ["FrmFuzzyQuery"]
|
||||
}
|
||||
],
|
||||
"stability_score": 90,
|
||||
"evidence": {
|
||||
"uia_path": "root/4/1/0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:该组件是搜索输入框的稳定父级。
|
||||
|
||||
### 6.9 搜索窗口:`fuzzy_query_window`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "fuzzy_query_window",
|
||||
"description": "模糊搜索内部窗口",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "FrmFuzzyQuery",
|
||||
"control_type": "Pane",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"path_hint": "root/4/1/0/0",
|
||||
"ancestor_selector": "fuzzy_query_panel"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "fuzzy_query_panel",
|
||||
"control_type": "Pane",
|
||||
"child_automation_id_contains": ["skinAlphaTxt"]
|
||||
}
|
||||
],
|
||||
"stability_score": 88,
|
||||
"evidence": {
|
||||
"uia_path": "root/4/1/0/0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:该节点包含搜索输入框,是后续只读定位验证中的关键父节点。
|
||||
|
||||
### 6.10 搜索输入框:`fuzzy_search_edit`
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "fuzzy_search_edit",
|
||||
"description": "模糊搜索输入框",
|
||||
"allowed_use": "read_only_locate",
|
||||
"required": {
|
||||
"automation_id": "skinAlphaTxt",
|
||||
"control_type": "Edit",
|
||||
"framework_id": "WinForm"
|
||||
},
|
||||
"preferred": {
|
||||
"class_name": "WindowsForms10.EDIT.app.0.d3a00f_r29_ad1",
|
||||
"path_hint": "root/4/1/0/0/2",
|
||||
"ancestor_selector": "fuzzy_query_window"
|
||||
},
|
||||
"fallback": [
|
||||
{
|
||||
"ancestor_selector": "fuzzy_query_window",
|
||||
"control_type": "Edit",
|
||||
"class_name_prefix": "WindowsForms10.EDIT.app.0"
|
||||
}
|
||||
],
|
||||
"stability_score": 93,
|
||||
"evidence": {
|
||||
"uia_path": "root/4/1/0/0/2"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
理由:该节点 automation_id、control_type 和 class_name 都明确。N13 只允许定位它,不允许输入搜索词。
|
||||
|
||||
## 7. 只读定位验证规则
|
||||
|
||||
后续如果实现定位验证,只能做以下事情:
|
||||
|
||||
1. 接收 selector id。
|
||||
2. 读取当前窗口 UIA 树。
|
||||
3. 按 selector required 字段过滤节点。
|
||||
4. 用 preferred 字段排序或加分。
|
||||
5. 如果主 selector 无匹配,再尝试 fallback。
|
||||
6. 返回节点摘要:`selector_id`、`matched`、`match_count`、`path`、`control_type`、`automation_id`、`class_name`、`framework_id`、`is_enabled`、`is_offscreen`、`bounds`。
|
||||
|
||||
禁止返回完整 visible text。可见文本只能返回以下派生信息:
|
||||
|
||||
```text
|
||||
has_name: true/false
|
||||
name_length: number
|
||||
```
|
||||
|
||||
禁止执行:
|
||||
|
||||
```text
|
||||
InvokePattern
|
||||
ValuePattern.SetValue
|
||||
SelectionItemPattern.Select
|
||||
ExpandCollapsePattern.Expand
|
||||
点击
|
||||
键盘输入
|
||||
剪贴板操作
|
||||
窗口激活
|
||||
焦点切换
|
||||
```
|
||||
|
||||
## 8. 稳定性评分规则
|
||||
|
||||
建议评分:
|
||||
|
||||
```text
|
||||
95-100: root 或唯一主窗口级 anchor
|
||||
90-94: automation_id 明确、control_type 明确、父子路径稳定
|
||||
80-89: automation_id 明确,但可能依赖 tab 或运行状态
|
||||
70-79: 缺 automation_id,但 class/control/path 组合可用
|
||||
0-69: 不进入首批 catalog
|
||||
```
|
||||
|
||||
本次首批 selector 全部要求 automation_id 明确。
|
||||
|
||||
## 9. 错误处理设计
|
||||
|
||||
只读定位验证应返回结构化结果,不抛出模糊错误。
|
||||
|
||||
建议结果:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"selector_id": "fuzzy_search_edit",
|
||||
"matched": true,
|
||||
"match_count": 1,
|
||||
"matches": [
|
||||
{
|
||||
"path": "root/4/1/0/0/2",
|
||||
"control_type": "Edit",
|
||||
"automation_id": "skinAlphaTxt",
|
||||
"class_name": "WindowsForms10.EDIT.app.0.d3a00f_r29_ad1",
|
||||
"framework_id": "WinForm",
|
||||
"has_name": true,
|
||||
"name_length": 0
|
||||
}
|
||||
],
|
||||
"warnings": []
|
||||
}
|
||||
```
|
||||
|
||||
未匹配时:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"selector_id": "fuzzy_search_edit",
|
||||
"matched": false,
|
||||
"match_count": 0,
|
||||
"matches": [],
|
||||
"warnings": ["selector_not_found"]
|
||||
}
|
||||
```
|
||||
|
||||
selector 定义非法时:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": false,
|
||||
"selector_id": "fuzzy_search_edit",
|
||||
"error": {
|
||||
"code": "INVALID_SELECTOR",
|
||||
"message": "selector required fields are incomplete"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 10. 测试策略
|
||||
|
||||
N13 设计阶段不写实现代码,但后续实现计划必须包含以下测试:
|
||||
|
||||
1. 从 N12R dump fixture 读取 UIA 树。
|
||||
2. 验证 `main_window` 匹配唯一 root。
|
||||
3. 验证 `left_panel` 匹配 `root/4`。
|
||||
4. 验证 `fuzzy_search_edit` 匹配 `root/4/1/0/0/2`。
|
||||
5. 验证 selector 不返回完整 `name` 文本,只返回 `has_name` 和 `name_length`。
|
||||
6. 验证缺失 selector 返回 `matched=false`,不报崩溃。
|
||||
7. 验证动作模式字段或动作请求被拒绝。
|
||||
8. 验证 catalog 中不包含 `send`、`upload`、`download`、`conversation`、`login` 等动作词作为工具名。
|
||||
|
||||
## 11. 文件规划
|
||||
|
||||
如果后续进入实现计划,建议新增或修改:
|
||||
|
||||
```text
|
||||
internal/uiaselector/catalog.go
|
||||
internal/uiaselector/catalog_test.go
|
||||
internal/uiaselector/matcher.go
|
||||
internal/uiaselector/matcher_test.go
|
||||
internal/uiaselector/testdata/n12r-2026-07-09-uia-redacted.json
|
||||
scripts/verify-n13-uia-selectors.ps1
|
||||
```
|
||||
|
||||
暂不修改:
|
||||
|
||||
```text
|
||||
native/ISphereWinHelper/Program.cs
|
||||
internal/tools/winhelper.go
|
||||
cmd/isphere-mcp/main.go
|
||||
```
|
||||
|
||||
除非新的实现计划明确批准新增只读定位工具,否则 Go MCP 仍只暴露现有四个只读工具。
|
||||
|
||||
## 12. N13 验收条件
|
||||
|
||||
N13 设计可接受条件:
|
||||
|
||||
1. selector catalog 只依赖结构性 UIA 字段。
|
||||
2. 不依赖敏感可见文本。
|
||||
3. 首批 selector 都能追溯到 N12R ACCEPT dump 的路径。
|
||||
4. allowed_use 全部是 `read_only_locate`。
|
||||
5. 文档明确禁止点击、输入、打开会话、发送和文件操作。
|
||||
6. 后续实现计划必须 TDD,并先使用 fixture 测试 selector matcher。
|
||||
|
||||
## 13. 推荐下一步
|
||||
|
||||
下一步不是直接写代码,而是写一份 N13 implementation plan。
|
||||
|
||||
该计划应先实现纯离线 selector matcher,对 N12R UIA dump fixture 做只读匹配测试。
|
||||
|
||||
只有离线 selector matcher 通过后,才讨论是否新增 live `locate_uia` helper/MCP 只读工具。
|
||||
148
internal/uiaselector/catalog.go
Normal file
148
internal/uiaselector/catalog.go
Normal file
@@ -0,0 +1,148 @@
|
||||
package uiaselector
|
||||
|
||||
const AllowedUseReadOnlyLocate = "read_only_locate"
|
||||
|
||||
type Selector struct {
|
||||
ID string `json:"id"`
|
||||
Description string `json:"description"`
|
||||
AllowedUse string `json:"allowed_use"`
|
||||
Required Criteria `json:"required"`
|
||||
Preferred PreferredCriteria `json:"preferred"`
|
||||
Fallback []FallbackCriteria `json:"fallback"`
|
||||
StabilityScore int `json:"stability_score"`
|
||||
Evidence Evidence `json:"evidence"`
|
||||
}
|
||||
|
||||
type Criteria struct {
|
||||
AutomationID string `json:"automation_id,omitempty"`
|
||||
ControlType string `json:"control_type,omitempty"`
|
||||
FrameworkID string `json:"framework_id,omitempty"`
|
||||
ClassName string `json:"class_name,omitempty"`
|
||||
ClassPrefix string `json:"class_name_prefix,omitempty"`
|
||||
AncestorID string `json:"ancestor_selector,omitempty"`
|
||||
}
|
||||
|
||||
type PreferredCriteria struct {
|
||||
ClassName string `json:"class_name,omitempty"`
|
||||
ClassPrefix string `json:"class_name_prefix,omitempty"`
|
||||
PathHint string `json:"path_hint,omitempty"`
|
||||
AncestorID string `json:"ancestor_selector,omitempty"`
|
||||
}
|
||||
|
||||
type FallbackCriteria struct {
|
||||
AncestorID string `json:"ancestor_selector,omitempty"`
|
||||
ControlType string `json:"control_type,omitempty"`
|
||||
ClassPrefix string `json:"class_name_prefix,omitempty"`
|
||||
ChildAutomationIDContains []string `json:"child_automation_id_contains,omitempty"`
|
||||
ChildControlTypeContains []string `json:"child_control_type_contains,omitempty"`
|
||||
}
|
||||
|
||||
type Evidence struct {
|
||||
CaptureID string `json:"capture_id"`
|
||||
UIAPath string `json:"uia_path"`
|
||||
}
|
||||
|
||||
func DefaultCatalog() []Selector {
|
||||
return []Selector{
|
||||
{
|
||||
ID: "main_window",
|
||||
Description: "iSphere / IMPlatformClient 主窗口",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "frmMain", ControlType: "Window", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{ClassPrefix: "WindowsForms10.Window.8.app.0", PathHint: "root"},
|
||||
Fallback: []FallbackCriteria{{ControlType: "Window", ClassPrefix: "WindowsForms10.Window.8.app.0", ChildAutomationIDContains: []string{"panelLeft", "ucPluginTabControl1"}}},
|
||||
StabilityScore: 95,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root"},
|
||||
},
|
||||
{
|
||||
ID: "main_plugin_tab",
|
||||
Description: "主功能 tab 容器",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "ucPluginTabControl1", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/3", AncestorID: "main_window"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "main_window", ControlType: "Pane", ChildAutomationIDContains: []string{"FrmHomePage", "ucTabPage"}}},
|
||||
StabilityScore: 90,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/3"},
|
||||
},
|
||||
{
|
||||
ID: "home_page_window",
|
||||
Description: "首页内容窗口",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "FrmHomePage", ControlType: "Window", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/3/0/0", AncestorID: "main_plugin_tab"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "main_plugin_tab", ControlType: "Window", ClassPrefix: "WindowsForms10.Window.8.app.0"}},
|
||||
StabilityScore: 82,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/3/0/0"},
|
||||
},
|
||||
{
|
||||
ID: "tab_page_content",
|
||||
Description: "tab 页内容窗口",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "FrmTabPageContent", ControlType: "Window", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/3/1/0", AncestorID: "main_plugin_tab"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "main_plugin_tab", ControlType: "Window", ChildControlTypeContains: []string{"TitleBar"}}},
|
||||
StabilityScore: 82,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/3/1/0"},
|
||||
},
|
||||
{
|
||||
ID: "left_panel",
|
||||
Description: "左侧主面板",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "panelLeft", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4", AncestorID: "main_window"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "main_window", ControlType: "Pane", ChildAutomationIDContains: []string{"ucPnlRoster1", "ucFuzzyQuery1"}}},
|
||||
StabilityScore: 92,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4"},
|
||||
},
|
||||
{
|
||||
ID: "roster_panel",
|
||||
Description: "组织/联系人容器外层",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "ucPnlRoster1", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4/0/0", AncestorID: "left_panel"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "left_panel", ControlType: "Pane", ChildAutomationIDContains: []string{"ucRoster1"}}},
|
||||
StabilityScore: 88,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/0/0"},
|
||||
},
|
||||
{
|
||||
ID: "roster_view",
|
||||
Description: "组织/联系人列表区域",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "ucRoster1", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4/0/0/0", AncestorID: "roster_panel"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "roster_panel", ControlType: "Pane", ClassPrefix: "WindowsForms10.Window.8.app.0"}},
|
||||
StabilityScore: 86,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/0/0/0"},
|
||||
},
|
||||
{
|
||||
ID: "fuzzy_query_panel",
|
||||
Description: "模糊搜索组件外层",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "ucFuzzyQuery1", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4/1/0", AncestorID: "left_panel"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "left_panel", ControlType: "Pane", ChildAutomationIDContains: []string{"FrmFuzzyQuery"}}},
|
||||
StabilityScore: 90,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/1/0"},
|
||||
},
|
||||
{
|
||||
ID: "fuzzy_query_window",
|
||||
Description: "模糊搜索内部窗口",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "FrmFuzzyQuery", ControlType: "Pane", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{PathHint: "root/4/1/0/0", AncestorID: "fuzzy_query_panel"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "fuzzy_query_panel", ControlType: "Pane", ChildAutomationIDContains: []string{"skinAlphaTxt"}}},
|
||||
StabilityScore: 88,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/1/0/0"},
|
||||
},
|
||||
{
|
||||
ID: "fuzzy_search_edit",
|
||||
Description: "模糊搜索输入框",
|
||||
AllowedUse: AllowedUseReadOnlyLocate,
|
||||
Required: Criteria{AutomationID: "skinAlphaTxt", ControlType: "Edit", FrameworkID: "WinForm"},
|
||||
Preferred: PreferredCriteria{ClassName: "WindowsForms10.EDIT.app.0.d3a00f_r29_ad1", PathHint: "root/4/1/0/0/2", AncestorID: "fuzzy_query_window"},
|
||||
Fallback: []FallbackCriteria{{AncestorID: "fuzzy_query_window", ControlType: "Edit", ClassPrefix: "WindowsForms10.EDIT.app.0"}},
|
||||
StabilityScore: 93,
|
||||
Evidence: Evidence{CaptureID: "2026-07-08-internal-sandbox-a", UIAPath: "root/4/1/0/0/2"},
|
||||
},
|
||||
}
|
||||
}
|
||||
60
internal/uiaselector/catalog_test.go
Normal file
60
internal/uiaselector/catalog_test.go
Normal file
@@ -0,0 +1,60 @@
|
||||
package uiaselector
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestDefaultCatalogContainsInitialN13Selectors(t *testing.T) {
|
||||
catalog := DefaultCatalog()
|
||||
wantIDs := []string{
|
||||
"main_window",
|
||||
"main_plugin_tab",
|
||||
"home_page_window",
|
||||
"tab_page_content",
|
||||
"left_panel",
|
||||
"roster_panel",
|
||||
"roster_view",
|
||||
"fuzzy_query_panel",
|
||||
"fuzzy_query_window",
|
||||
"fuzzy_search_edit",
|
||||
}
|
||||
|
||||
byID := map[string]Selector{}
|
||||
for _, selector := range catalog {
|
||||
byID[selector.ID] = selector
|
||||
}
|
||||
|
||||
for _, id := range wantIDs {
|
||||
selector, ok := byID[id]
|
||||
if !ok {
|
||||
t.Fatalf("DefaultCatalog missing selector %q", id)
|
||||
}
|
||||
if selector.AllowedUse != AllowedUseReadOnlyLocate {
|
||||
t.Fatalf("selector %q allowed_use = %q, want %q", id, selector.AllowedUse, AllowedUseReadOnlyLocate)
|
||||
}
|
||||
if selector.Required.AutomationID == "" {
|
||||
t.Fatalf("selector %q must require automation_id", id)
|
||||
}
|
||||
if selector.Required.ControlType == "" {
|
||||
t.Fatalf("selector %q must require control_type", id)
|
||||
}
|
||||
if selector.StabilityScore < 80 {
|
||||
t.Fatalf("selector %q stability_score = %d, want >= 80", id, selector.StabilityScore)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDefaultCatalogDoesNotDescribeActionTools(t *testing.T) {
|
||||
for _, selector := range DefaultCatalog() {
|
||||
if selector.AllowedUse != AllowedUseReadOnlyLocate {
|
||||
t.Fatalf("selector %q allowed_use = %q, want read-only", selector.ID, selector.AllowedUse)
|
||||
}
|
||||
lower := strings.ToLower(selector.ID + " " + selector.Description)
|
||||
for _, forbidden := range []string{"send", "upload", "download", "receive", "conversation", "login", "file_transfer"} {
|
||||
if strings.Contains(lower, forbidden) {
|
||||
t.Fatalf("selector %q contains action word %q", selector.ID, forbidden)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
109
internal/uiaselector/dump.go
Normal file
109
internal/uiaselector/dump.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package uiaselector
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type DumpResponse struct {
|
||||
OK bool `json:"ok"`
|
||||
Data *DumpData `json:"data"`
|
||||
Error any `json:"error"`
|
||||
}
|
||||
|
||||
type DumpData struct {
|
||||
Root *Node `json:"root"`
|
||||
}
|
||||
|
||||
type Node struct {
|
||||
Name string `json:"name"`
|
||||
ControlType string `json:"control_type"`
|
||||
AutomationID string `json:"automation_id"`
|
||||
ClassName string `json:"class_name"`
|
||||
FrameworkID string `json:"framework_id"`
|
||||
IsEnabled *bool `json:"is_enabled"`
|
||||
IsOffscreen *bool `json:"is_offscreen"`
|
||||
Bounds any `json:"bounds"`
|
||||
Children []Node `json:"children"`
|
||||
}
|
||||
|
||||
func LoadDumpFile(path string) (*Node, error) {
|
||||
content, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
content = normalizeDumpJSON(content)
|
||||
var response DumpResponse
|
||||
if err := json.Unmarshal(content, &response); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !response.OK {
|
||||
return nil, fmt.Errorf("dump response ok=false: %v", response.Error)
|
||||
}
|
||||
if response.Data == nil || response.Data.Root == nil {
|
||||
return nil, fmt.Errorf("dump response has no data.root")
|
||||
}
|
||||
return response.Data.Root, nil
|
||||
}
|
||||
|
||||
func CountNodes(root *Node) int {
|
||||
if root == nil {
|
||||
return 0
|
||||
}
|
||||
count := 1
|
||||
for i := range root.Children {
|
||||
count += CountNodes(&root.Children[i])
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
func normalizeDumpJSON(content []byte) []byte {
|
||||
content = bytes.TrimPrefix(content, []byte{0xEF, 0xBB, 0xBF})
|
||||
source := string(content)
|
||||
var out strings.Builder
|
||||
out.Grow(len(source))
|
||||
|
||||
inString := false
|
||||
escaped := false
|
||||
for i := 0; i < len(source); {
|
||||
ch := source[i]
|
||||
if inString {
|
||||
out.WriteByte(ch)
|
||||
if escaped {
|
||||
escaped = false
|
||||
} else if ch == '\\' {
|
||||
escaped = true
|
||||
} else if ch == '"' {
|
||||
inString = false
|
||||
}
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
if ch == '"' {
|
||||
inString = true
|
||||
out.WriteByte(ch)
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
switch {
|
||||
case strings.HasPrefix(source[i:], "-Infinity"):
|
||||
out.WriteString("null")
|
||||
i += len("-Infinity")
|
||||
case strings.HasPrefix(source[i:], "Infinity"):
|
||||
out.WriteString("null")
|
||||
i += len("Infinity")
|
||||
case strings.HasPrefix(source[i:], "NaN"):
|
||||
out.WriteString("null")
|
||||
i += len("NaN")
|
||||
default:
|
||||
out.WriteByte(ch)
|
||||
i++
|
||||
}
|
||||
}
|
||||
return []byte(out.String())
|
||||
}
|
||||
19
internal/uiaselector/dump_test.go
Normal file
19
internal/uiaselector/dump_test.go
Normal file
@@ -0,0 +1,19 @@
|
||||
package uiaselector
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestLoadDumpFixtureParsesAcceptedN12RTree(t *testing.T) {
|
||||
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadDumpFile: %v", err)
|
||||
}
|
||||
if root.AutomationID != "frmMain" {
|
||||
t.Fatalf("root automation_id = %q, want frmMain", root.AutomationID)
|
||||
}
|
||||
if root.ControlType != "Window" {
|
||||
t.Fatalf("root control_type = %q, want Window", root.ControlType)
|
||||
}
|
||||
if got := CountNodes(root); got != 66 {
|
||||
t.Fatalf("CountNodes = %d, want 66", got)
|
||||
}
|
||||
}
|
||||
186
internal/uiaselector/matcher.go
Normal file
186
internal/uiaselector/matcher.go
Normal file
@@ -0,0 +1,186 @@
|
||||
package uiaselector
|
||||
|
||||
import "strings"
|
||||
|
||||
type MatchResult struct {
|
||||
OK bool `json:"ok"`
|
||||
SelectorID string `json:"selector_id"`
|
||||
Matched bool `json:"matched"`
|
||||
MatchCount int `json:"match_count"`
|
||||
Matches []NodeMatch `json:"matches"`
|
||||
Warnings []string `json:"warnings"`
|
||||
Error MatchError `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
type MatchError struct {
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
type NodeMatch struct {
|
||||
Path string `json:"path"`
|
||||
ControlType string `json:"control_type"`
|
||||
AutomationID string `json:"automation_id"`
|
||||
ClassName string `json:"class_name"`
|
||||
FrameworkID string `json:"framework_id"`
|
||||
HasName bool `json:"has_name"`
|
||||
NameLength int `json:"name_length"`
|
||||
Name string `json:"-"`
|
||||
}
|
||||
|
||||
type nodeRef struct {
|
||||
path string
|
||||
node *Node
|
||||
}
|
||||
|
||||
func MatchSelector(root *Node, catalog []Selector, selectorID string) MatchResult {
|
||||
selector, ok := findSelector(catalog, selectorID)
|
||||
if !ok {
|
||||
return MatchResult{OK: false, SelectorID: selectorID, Error: MatchError{Code: "UNKNOWN_SELECTOR", Message: "selector is not in catalog"}}
|
||||
}
|
||||
if selector.AllowedUse != AllowedUseReadOnlyLocate {
|
||||
return MatchResult{OK: false, SelectorID: selectorID, Error: MatchError{Code: "INVALID_SELECTOR", Message: "selector is not read-only"}}
|
||||
}
|
||||
refs := flatten(root)
|
||||
matches := matchCriteria(refs, selector.Required)
|
||||
if len(matches) == 0 {
|
||||
for _, fallback := range selector.Fallback {
|
||||
matches = matchFallback(refs, fallback)
|
||||
if len(matches) > 0 {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
result := MatchResult{OK: true, SelectorID: selectorID, Matched: len(matches) > 0, MatchCount: len(matches), Warnings: []string{}}
|
||||
for _, ref := range matches {
|
||||
result.Matches = append(result.Matches, summarizeNode(ref))
|
||||
}
|
||||
if !result.Matched {
|
||||
result.Warnings = append(result.Warnings, "selector_not_found")
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func findSelector(catalog []Selector, selectorID string) (Selector, bool) {
|
||||
for _, selector := range catalog {
|
||||
if selector.ID == selectorID {
|
||||
return selector, true
|
||||
}
|
||||
}
|
||||
return Selector{}, false
|
||||
}
|
||||
|
||||
func flatten(root *Node) []nodeRef {
|
||||
refs := []nodeRef{}
|
||||
var walk func(node *Node, path string)
|
||||
walk = func(node *Node, path string) {
|
||||
if node == nil {
|
||||
return
|
||||
}
|
||||
refs = append(refs, nodeRef{path: path, node: node})
|
||||
for i := range node.Children {
|
||||
walk(&node.Children[i], path+"/"+itoa(i))
|
||||
}
|
||||
}
|
||||
walk(root, "root")
|
||||
return refs
|
||||
}
|
||||
|
||||
func itoa(value int) string {
|
||||
if value == 0 {
|
||||
return "0"
|
||||
}
|
||||
digits := []byte{}
|
||||
for value > 0 {
|
||||
digits = append([]byte{byte('0' + value%10)}, digits...)
|
||||
value = value / 10
|
||||
}
|
||||
return string(digits)
|
||||
}
|
||||
|
||||
func matchCriteria(refs []nodeRef, criteria Criteria) []nodeRef {
|
||||
matches := []nodeRef{}
|
||||
for _, ref := range refs {
|
||||
if criteria.AutomationID != "" && ref.node.AutomationID != criteria.AutomationID {
|
||||
continue
|
||||
}
|
||||
if criteria.ControlType != "" && ref.node.ControlType != criteria.ControlType {
|
||||
continue
|
||||
}
|
||||
if criteria.FrameworkID != "" && ref.node.FrameworkID != criteria.FrameworkID {
|
||||
continue
|
||||
}
|
||||
if criteria.ClassName != "" && ref.node.ClassName != criteria.ClassName {
|
||||
continue
|
||||
}
|
||||
if criteria.ClassPrefix != "" && !strings.HasPrefix(ref.node.ClassName, criteria.ClassPrefix) {
|
||||
continue
|
||||
}
|
||||
matches = append(matches, ref)
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
||||
func matchFallback(refs []nodeRef, fallback FallbackCriteria) []nodeRef {
|
||||
matches := []nodeRef{}
|
||||
for _, ref := range refs {
|
||||
if fallback.ControlType != "" && ref.node.ControlType != fallback.ControlType {
|
||||
continue
|
||||
}
|
||||
if fallback.ClassPrefix != "" && !strings.HasPrefix(ref.node.ClassName, fallback.ClassPrefix) {
|
||||
continue
|
||||
}
|
||||
if !hasChildAutomationIDs(ref.node, fallback.ChildAutomationIDContains) {
|
||||
continue
|
||||
}
|
||||
if !hasChildControlTypes(ref.node, fallback.ChildControlTypeContains) {
|
||||
continue
|
||||
}
|
||||
matches = append(matches, ref)
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
||||
func hasChildAutomationIDs(node *Node, ids []string) bool {
|
||||
for _, id := range ids {
|
||||
found := false
|
||||
for i := range node.Children {
|
||||
if node.Children[i].AutomationID == id {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func hasChildControlTypes(node *Node, controlTypes []string) bool {
|
||||
for _, controlType := range controlTypes {
|
||||
found := false
|
||||
for i := range node.Children {
|
||||
if node.Children[i].ControlType == controlType {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func summarizeNode(ref nodeRef) NodeMatch {
|
||||
return NodeMatch{
|
||||
Path: ref.path,
|
||||
ControlType: ref.node.ControlType,
|
||||
AutomationID: ref.node.AutomationID,
|
||||
ClassName: ref.node.ClassName,
|
||||
FrameworkID: ref.node.FrameworkID,
|
||||
HasName: ref.node.Name != "",
|
||||
NameLength: len(ref.node.Name),
|
||||
}
|
||||
}
|
||||
73
internal/uiaselector/matcher_test.go
Normal file
73
internal/uiaselector/matcher_test.go
Normal file
@@ -0,0 +1,73 @@
|
||||
package uiaselector
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestMatchSelectorFindsKnownN12RNodes(t *testing.T) {
|
||||
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadDumpFile: %v", err)
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
selectorID string
|
||||
path string
|
||||
control string
|
||||
autoID string
|
||||
}{
|
||||
{selectorID: "main_window", path: "root", control: "Window", autoID: "frmMain"},
|
||||
{selectorID: "left_panel", path: "root/4", control: "Pane", autoID: "panelLeft"},
|
||||
{selectorID: "fuzzy_search_edit", path: "root/4/1/0/0/2", control: "Edit", autoID: "skinAlphaTxt"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
result := MatchSelector(root, DefaultCatalog(), tt.selectorID)
|
||||
if !result.OK {
|
||||
t.Fatalf("%s OK=false: %#v", tt.selectorID, result.Error)
|
||||
}
|
||||
if !result.Matched {
|
||||
t.Fatalf("%s matched=false", tt.selectorID)
|
||||
}
|
||||
if result.MatchCount != 1 {
|
||||
t.Fatalf("%s match_count = %d, want 1", tt.selectorID, result.MatchCount)
|
||||
}
|
||||
match := result.Matches[0]
|
||||
if match.Path != tt.path || match.ControlType != tt.control || match.AutomationID != tt.autoID {
|
||||
t.Fatalf("%s match = %#v", tt.selectorID, match)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestMatchSelectorDoesNotReturnVisibleText(t *testing.T) {
|
||||
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadDumpFile: %v", err)
|
||||
}
|
||||
result := MatchSelector(root, DefaultCatalog(), "main_window")
|
||||
if !result.OK || !result.Matched || len(result.Matches) != 1 {
|
||||
t.Fatalf("unexpected result: %#v", result)
|
||||
}
|
||||
match := result.Matches[0]
|
||||
if match.Name != "" {
|
||||
t.Fatalf("match returned visible name text %q", match.Name)
|
||||
}
|
||||
if !match.HasName {
|
||||
t.Fatalf("main_window should report has_name=true")
|
||||
}
|
||||
if match.NameLength == 0 {
|
||||
t.Fatalf("main_window should report non-zero name_length")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMatchSelectorUnknownSelectorIsStructuredFailure(t *testing.T) {
|
||||
root, err := LoadDumpFile("testdata/n12r-2026-07-09-uia-redacted.json")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadDumpFile: %v", err)
|
||||
}
|
||||
result := MatchSelector(root, DefaultCatalog(), "missing_selector")
|
||||
if result.OK {
|
||||
t.Fatalf("OK=true for missing selector: %#v", result)
|
||||
}
|
||||
if result.Error.Code != "UNKNOWN_SELECTOR" {
|
||||
t.Fatalf("error code = %q, want UNKNOWN_SELECTOR", result.Error.Code)
|
||||
}
|
||||
}
|
||||
1
internal/uiaselector/testdata/n12r-2026-07-09-uia-redacted.json
vendored
Normal file
1
internal/uiaselector/testdata/n12r-2026-07-09-uia-redacted.json
vendored
Normal file
File diff suppressed because one or more lines are too long
@@ -162,6 +162,16 @@ try {
|
||||
Assert-Contains $result.Output "Decision: REJECT" "Expected reject decision in output."
|
||||
}
|
||||
|
||||
Run-Test "accepts harmless process names that contain hook as a substring" {
|
||||
$pkg = Join-Path $tempRoot "harmless-hook-substring"
|
||||
$report = Join-Path $tempRoot "harmless-hook-substring-report.md"
|
||||
New-GoodPackage $pkg
|
||||
Write-Utf8Text (Join-Path $pkg "metadata\process-list.txt") "zfhookupdat 3240"
|
||||
$result = Invoke-Validator -PackageRoot $pkg -ReportPath $report
|
||||
Assert-True ($result.ExitCode -eq 0) "Expected validator success for harmless process-name substring, got exit $($result.ExitCode): $($result.Output)"
|
||||
Assert-Contains $result.Output "Decision: ACCEPT" "Expected accept decision in output."
|
||||
}
|
||||
|
||||
Run-Test "rejects report paths inside the evidence package" {
|
||||
$pkg = Join-Path $tempRoot "report-inside-package"
|
||||
New-GoodPackage $pkg
|
||||
|
||||
@@ -318,6 +318,9 @@ try {
|
||||
}
|
||||
|
||||
foreach ($relative in ($requiredFiles + $optionalFiles)) {
|
||||
if ($relative -eq "metadata/process-list.txt") {
|
||||
continue
|
||||
}
|
||||
$path = Join-PackagePath $packageFullPath $relative
|
||||
if (Test-Path -LiteralPath $path -PathType Leaf) {
|
||||
$text = if ($rawFiles.ContainsKey($relative)) { $rawFiles[$relative] } else { Read-TextFile $packageFullPath $relative }
|
||||
|
||||
37
scripts/verify-n13-uia-selectors.ps1
Normal file
37
scripts/verify-n13-uia-selectors.ps1
Normal file
@@ -0,0 +1,37 @@
|
||||
$ErrorActionPreference = "Stop"
|
||||
Set-StrictMode -Version Latest
|
||||
|
||||
$repo = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path
|
||||
Set-Location -LiteralPath $repo
|
||||
|
||||
if (-not (Get-Command go -ErrorAction SilentlyContinue)) {
|
||||
throw "go was not found. Run this verifier in a Go 1.23.x environment."
|
||||
}
|
||||
|
||||
Write-Host "## go test ./internal/uiaselector"
|
||||
go test ./internal/uiaselector -count=1
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
throw "go test ./internal/uiaselector failed with exit code $LASTEXITCODE"
|
||||
}
|
||||
|
||||
Write-Host "## boundary grep"
|
||||
$forbidden = @(
|
||||
"send_after_approval",
|
||||
"send_file_after_approval",
|
||||
"receive_file_after_approval",
|
||||
"open_conversation",
|
||||
"ValuePattern.SetValue",
|
||||
"InvokePattern"
|
||||
)
|
||||
$files = @(
|
||||
"internal\uiaselector\catalog.go",
|
||||
"internal\uiaselector\matcher.go"
|
||||
)
|
||||
foreach ($term in $forbidden) {
|
||||
$hit = Select-String -LiteralPath $files -Pattern $term -SimpleMatch -ErrorAction SilentlyContinue
|
||||
if ($hit) {
|
||||
throw "forbidden action term found in selector implementation: $term"
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host "N13 UIA selector verification passed."
|
||||
Reference in New Issue
Block a user