test: add file download resolver v2 diagnostic

This commit is contained in:
zhaoyilun
2026-07-11 00:20:15 +08:00
parent 7db022469c
commit c8b619c254
5 changed files with 310 additions and 22 deletions

View File

@@ -59,7 +59,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme
## Current loop
Current loop: `R7 send-file preview tool complete`; next R8 send-file idempotency and audit hardening.
Current loop: `R10 receive-file download resolver v2 diagnostic complete`; next R11 receive-file download preview contract.
Active continuous execution plan: `docs/superpowers/plans/2026-07-10-r6f-r14-continuous-execution-plan.md`.
@@ -240,8 +240,8 @@ Continuous execution plan R6f-R14 is approved and execution has started:
- Plan file: `docs/superpowers/plans/2026-07-10-r6f-r14-continuous-execution-plan.md`.
- Scope: 15 ordered rounds covering send-message connector/gate hardening, send-file preview/idempotency/package, receive-file download preview, receive-message reconciliation, end-to-end business smoke, and release-candidate report.
- Execution rule after approval: one round at a time, status-card update, verification, commit, push, then continue automatically until an evidence-only blocker requires user action.
- Last completed round: R9 send-file upload evidence package.
- Next active round: R10 receive-file download resolver v2 diagnostic.
- Last completed round: R10 receive-file download resolver v2 diagnostic.
- Next active round: R11 receive-file download preview contract.
R6g send audit and idempotency replay diagnostics is complete:
@@ -326,3 +326,12 @@ R9 send-file upload evidence package is complete:
- Local verification generated and validated `runs\send-file-sandbox-gate-package.zip`; no production file upload connector was enabled.
- Verification passed: git diff --check, scripts\verify-send-file-sandbox-gate-package.ps1, go test ./..., go build ./cmd/isphere-mcp, and scripts\verify-go-mcp.ps1.
- Next node: R10 receive-file download resolver v2 diagnostic.
R10 receive-file download resolver v2 diagnostic is complete:
- Added fixture-backed scoring to `scripts\verify-file-cache-mapping.ps1` with `-FixtureRecordsPath`, `-CacheRoot`, and `-SanitizedOutputPath`.
- `scripts\test-verify-file-cache-mapping.ps1` now covers the legacy archive-list summary and the v2 scoring fixture.
- Fixture result: `records_checked=3`, `cache_candidates_checked=4`, `accepted_matches=2`, `ambiguous_matches=0`, `score_100_matches=1`, `score_80_matches=1`, and `score_40_only_matches=1`.
- Diagnostic output keeps `raw_paths_printed=false`, `file_contents_read=false`, and `file_paths_returned=false`.
- Decision: resolver logic is proven, but real production file download remains blocked until a real cache mapping package produces accepted non-ambiguous matches.
- Next node: R11 receive-file download preview contract.

View File

@@ -0,0 +1,49 @@
# File download mapping v2 diagnostic - 2026-07-10
## Purpose
R10 checks whether received-file metadata can be mapped to an operator-local cache/download file strongly enough for a future `isphere_receive_files` download path.
## What changed
`scripts/verify-file-cache-mapping.ps1` now has a fixture-backed scoring mode:
- `-FixtureRecordsPath`: JSON records with `file_name`, `size_bytes`, `source_id`, and `created_at`.
- `-CacheRoot`: operator-local cache root or fixture cache root.
- `-SanitizedOutputPath`: writes a compact JSON summary without raw paths or file contents.
Scoring rules:
| Rule | Score | Accepted |
| --- | ---: | --- |
| exact filename + exact size + close timestamp | 100 | yes, if unique |
| exact size + source id appears in cache name/ref | 80 | yes, if unique |
| filename only | 40 | no |
## Fixture result
`powershell -NoProfile -ExecutionPolicy Bypass -File scripts\test-verify-file-cache-mapping.ps1` produced:
```json
{"ok":true,"archive_candidate_count":2,"records_checked":3,"cache_candidates_checked":4,"accepted_matches":2,"ambiguous_matches":0,"score_100_matches":1,"score_80_matches":1,"score_40_only_matches":1,"raw_paths_printed":false,"file_contents_read":false}
```
Interpretation:
- The resolver algorithm can produce accepted matches when file metadata is strong.
- One weak filename-only case is intentionally not accepted.
- The diagnostic output stays committed-safe: no raw operator paths and no file contents.
## Production decision
This round proves the resolver logic, not a real iSphere cache mapping on this machine. The local machine still cannot log in to iSphere, and no real returned receive-file package in this repository state proves a stable cache/download mapping.
Therefore:
- `isphere_receive_files` list mode remains usable.
- Real file download/copy remains blocked until a real cache mapping package produces accepted, non-ambiguous matches.
- R11 should implement a download preview/blocked contract so digital employees get machine-readable status instead of a generic failure.
## Next node
R11: `Receive-file download preview contract`.

View File

@@ -709,7 +709,7 @@ git push gitea main
- `-MsgLibDb` and `-SQLiteDll` accept copied DB/runtime paths when real copied DB is available; fixture tests do not require real DB files.
- `-SanitizedOutputPath` writes committed-safe JSON summary to a caller-selected output file.
- [ ] **Step 1: Add fixture test for resolver scoring**
- [x] **Step 1: Add fixture test for resolver scoring**
Fixture contains three DB-like file records and four cache-like file entries. Expected scoring:
@@ -719,15 +719,15 @@ exact size + source id in name/ref => score 80
filename only => score 40 and not accepted
```
- [ ] **Step 2: Implement scoring and sanitized output**
- [x] **Step 2: Implement scoring and sanitized output**
Output fields: `records_checked`, `cache_candidates_checked`, `accepted_matches`, `ambiguous_matches`, `raw_paths_printed=false`, `file_contents_read=false`.
- [ ] **Step 3: Write decision note**
- [x] **Step 3: Write decision note**
`docs/source-discovery/2026-07-10-file-download-mapping-v2.md` records whether accepted matches exist. If not, download remains blocked and UI/client download connector becomes the next route.
- [ ] **Step 4: Verify and commit**
- [x] **Step 4: Verify and commit**
```powershell
git diff --check
@@ -742,6 +742,14 @@ git push gitea main
**Acceptance gate:** 文件下载要么得到可实现映射,要么得到明确阻断证据。
**R10 Result:**
- Added v2 fixture scoring to `scripts/verify-file-cache-mapping.ps1`.
- Updated `scripts/test-verify-file-cache-mapping.ps1` to prove exact filename/size/time gets score 100, source-id/size gets score 80, and filename-only gets score 40 without acceptance.
- Fixture verification produced `records_checked=3`, `cache_candidates_checked=4`, `accepted_matches=2`, `ambiguous_matches=0`, `score_100_matches=1`, `score_80_matches=1`, `score_40_only_matches=1`, `raw_paths_printed=false`, and `file_contents_read=false`.
- Created `docs/source-discovery/2026-07-10-file-download-mapping-v2.md`.
- Real production file download remains blocked because this proves resolver logic, not a real iSphere cache mapping from an online package.
---
### R11: Receive-file download preview contract

View File

@@ -5,13 +5,22 @@ Set-StrictMode -Version Latest
$repo = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path
$script = Join-Path $repo "scripts\verify-file-cache-mapping.ps1"
$tempDir = Join-Path $repo ("runs\file-cache-mapping-test-" + [guid]::NewGuid().ToString("N"))
$tempDir = Join-Path ([System.IO.Path]::GetTempPath()) ("isphere-file-cache-mapping-test-" + [guid]::NewGuid().ToString("N"))
$archiveList = Join-Path $tempDir "archive-list.txt"
$cacheRoot = Join-Path $tempDir "cache-root"
$recordsPath = Join-Path $tempDir "fixture-records.json"
$summaryPath = Join-Path $tempDir "sanitized-summary.json"
function Assert-True([bool]$Condition, [string]$Message) {
if (-not $Condition) { throw $Message }
}
function Write-AsciiFile([string]$Path, [string]$Text) {
$parent = Split-Path -Parent $Path
New-Item -ItemType Directory -Force -Path $parent | Out-Null
[System.IO.File]::WriteAllBytes($Path, [System.Text.Encoding]::ASCII.GetBytes($Text))
}
try {
New-Item -ItemType Directory -Force -Path $tempDir | Out-Null
@(
@@ -20,21 +29,79 @@ try {
"C:\redacted\other\ignored.txt"
) | Set-Content -LiteralPath $archiveList -Encoding UTF8
$output = & powershell -NoProfile -ExecutionPolicy Bypass -File $script -ArchiveListPath $archiveList -SkipDb
$archiveOutput = & powershell -NoProfile -ExecutionPolicy Bypass -File $script -ArchiveListPath $archiveList -SkipDb
if ($LASTEXITCODE -ne 0) {
throw "verify-file-cache-mapping.ps1 failed with exit code $LASTEXITCODE"
throw "archive-list verify-file-cache-mapping.ps1 failed with exit code $LASTEXITCODE"
}
$json = $output | ConvertFrom-Json
$archiveJson = $archiveOutput | ConvertFrom-Json
Assert-True ($json.ok -eq $true) "ok should be true"
Assert-True ($json.cache_checked -eq $true) "cache should be checked"
Assert-True ($json.cache_candidate_count -eq 2) "cache candidate count should be 2"
Assert-True ($json.cache_extension_counts.".docx" -eq 1) "docx extension count should be 1"
Assert-True ($json.cache_extension_counts.".xlsx" -eq 1) "xlsx extension count should be 1"
Assert-True ($json.hash_dir_length_counts."32" -eq 2) "hash dir length count should be 2"
Assert-True ($json.raw_paths_printed -eq $false) "raw paths must not be printed"
Assert-True ($json.file_contents_read -eq $false) "file contents must not be read"
Assert-True ($json.file_paths_returned -eq $false) "file paths must not be returned"
Assert-True ($archiveJson.ok -eq $true) "archive ok should be true"
Assert-True ($archiveJson.cache_checked -eq $true) "archive cache should be checked"
Assert-True ($archiveJson.cache_candidate_count -eq 2) "archive cache candidate count should be 2"
Assert-True ($archiveJson.cache_extension_counts.".docx" -eq 1) "docx extension count should be 1"
Assert-True ($archiveJson.cache_extension_counts.".xlsx" -eq 1) "xlsx extension count should be 1"
Assert-True ($archiveJson.hash_dir_length_counts."32" -eq 2) "hash dir length count should be 2"
Assert-True ($archiveJson.raw_paths_printed -eq $false) "archive raw paths must not be printed"
Assert-True ($archiveJson.file_contents_read -eq $false) "archive file contents must not be read"
Assert-True ($archiveJson.file_paths_returned -eq $false) "archive file paths must not be returned"
$reportPath = Join-Path $cacheRoot "stable\report.docx"
$payloadPath = Join-Path $cacheRoot "source-src-abc-123\payload.bin"
$loosePath = Join-Path $cacheRoot "loose\loose.txt"
$noisePath = Join-Path $cacheRoot "noise\noise.tmp"
Write-AsciiFile $reportPath "hello"
Write-AsciiFile $payloadPath "payload"
Write-AsciiFile $loosePath "abc"
Write-AsciiFile $noisePath "noise"
(Get-Item -LiteralPath $reportPath).LastWriteTimeUtc = [datetime]"2026-07-10T04:00:10Z"
(Get-Item -LiteralPath $payloadPath).LastWriteTimeUtc = [datetime]"2026-07-10T05:00:00Z"
(Get-Item -LiteralPath $loosePath).LastWriteTimeUtc = [datetime]"2026-07-10T06:00:00Z"
(Get-Item -LiteralPath $noisePath).LastWriteTimeUtc = [datetime]"2026-07-10T07:00:00Z"
@(
[ordered]@{ file_name = "report.docx"; size_bytes = 5; source_id = "report-source"; created_at = "2026-07-10T04:00:00Z" },
[ordered]@{ file_name = "receipt.pdf"; size_bytes = 7; source_id = "src-abc-123"; created_at = "2026-07-10T04:10:00Z" },
[ordered]@{ file_name = "loose.txt"; size_bytes = 999; source_id = "loose-source"; created_at = "2026-07-10T04:20:00Z" }
) | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $recordsPath -Encoding UTF8
$fixtureOutput = & powershell -NoProfile -ExecutionPolicy Bypass -File $script -SkipDb -CacheRoot $cacheRoot -FixtureRecordsPath $recordsPath -SanitizedOutputPath $summaryPath
if ($LASTEXITCODE -ne 0) {
throw "fixture verify-file-cache-mapping.ps1 failed with exit code $LASTEXITCODE"
}
$fixtureJson = $fixtureOutput | ConvertFrom-Json
Assert-True ($fixtureJson.ok -eq $true) "fixture ok should be true"
Assert-True ($fixtureJson.records_checked -eq 3) "records_checked should be 3"
Assert-True ($fixtureJson.cache_candidates_checked -eq 4) "cache_candidates_checked should be 4"
Assert-True ($fixtureJson.accepted_matches -eq 2) "accepted_matches should be 2"
Assert-True ($fixtureJson.ambiguous_matches -eq 0) "ambiguous_matches should be 0"
Assert-True ($fixtureJson.score_100_matches -eq 1) "score_100_matches should be 1"
Assert-True ($fixtureJson.score_80_matches -eq 1) "score_80_matches should be 1"
Assert-True ($fixtureJson.score_40_only_matches -eq 1) "score_40_only_matches should be 1"
Assert-True ($fixtureJson.used_fixture_records -eq $true) "used_fixture_records should be true"
Assert-True ($fixtureJson.mapping_validated -eq $true) "fixture mapping should be validated"
Assert-True ($fixtureJson.raw_paths_printed -eq $false) "fixture raw paths must not be printed"
Assert-True ($fixtureJson.file_contents_read -eq $false) "fixture file contents must not be read"
Assert-True (Test-Path -LiteralPath $summaryPath) "sanitized summary should be written"
$summaryText = Get-Content -LiteralPath $summaryPath -Raw
Assert-True (-not $summaryText.Contains($cacheRoot)) "sanitized summary must not contain raw cache root"
Assert-True (-not $summaryText.Contains("hello")) "sanitized summary must not contain file content"
Assert-True (-not $summaryText.Contains("payload")) "sanitized summary must not contain file content"
[ordered]@{
ok = $true
archive_candidate_count = $archiveJson.cache_candidate_count
records_checked = $fixtureJson.records_checked
cache_candidates_checked = $fixtureJson.cache_candidates_checked
accepted_matches = $fixtureJson.accepted_matches
ambiguous_matches = $fixtureJson.ambiguous_matches
score_100_matches = $fixtureJson.score_100_matches
score_80_matches = $fixtureJson.score_80_matches
score_40_only_matches = $fixtureJson.score_40_only_matches
raw_paths_printed = $fixtureJson.raw_paths_printed
file_contents_read = $fixtureJson.file_contents_read
} | ConvertTo-Json -Depth 4 -Compress
}
finally {
if (Test-Path -LiteralPath $tempDir) {

View File

@@ -4,6 +4,8 @@ param(
[string]$SQLiteDllPath = $env:ISPHERE_MSGLIB_SQLITE_DLL,
[string]$DbPath = $env:ISPHERE_MSGLIB_DB,
[string]$Password = "123",
[string]$FixtureRecordsPath = "",
[string]$SanitizedOutputPath = "",
[switch]$SkipDb
)
@@ -116,6 +118,118 @@ function Read-CacheSummaryFromRoot([string]$Path) {
return $summary
}
function ConvertTo-SafeFileRecord($Value) {
$createdAt = $null
if (-not [string]::IsNullOrWhiteSpace([string]$Value.created_at)) {
[datetime]$parsed = [datetime]::MinValue
if ([datetime]::TryParse([string]$Value.created_at, [ref]$parsed)) {
$createdAt = $parsed.ToUniversalTime()
}
}
return [pscustomobject]@{
normalized_name = Normalize-FileName ([string]$Value.file_name)
size_bytes = if ($null -ne $Value.size_bytes) { [int64]$Value.size_bytes } else { -1 }
source_id = ([string]$Value.source_id).Trim().ToLowerInvariant()
created_at_utc = $createdAt
}
}
function Read-FixtureFileRecords([string]$Path) {
if (-not (Test-Path -LiteralPath $Path)) {
throw "FixtureRecordsPath not found"
}
$json = Get-Content -LiteralPath $Path -Raw | ConvertFrom-Json
$safeRecords = @()
foreach ($record in $json) {
$safeRecords += (ConvertTo-SafeFileRecord $record)
}
return $safeRecords
}
function Read-CacheCandidatesForScoring([string]$Path) {
if (-not (Test-Path -LiteralPath $Path)) {
throw "CacheRoot not found"
}
$candidates = @()
foreach ($file in Get-ChildItem -LiteralPath $Path -File -Recurse) {
$searchText = (($file.Name + " " + $file.DirectoryName) -replace '[\\/]', ' ').ToLowerInvariant()
$candidates += [pscustomobject]@{
normalized_name = Normalize-FileName $file.Name
size_bytes = [int64]$file.Length
last_write_utc = $file.LastWriteTimeUtc
search_text = $searchText
}
}
return $candidates
}
function Test-CloseTimestamp($Record, $Candidate) {
if ($null -eq $Record.created_at_utc -or $null -eq $Candidate.last_write_utc) {
return $false
}
$delta = [Math]::Abs((New-TimeSpan -Start $Record.created_at_utc -End $Candidate.last_write_utc).TotalSeconds)
return $delta -le 600
}
function Get-ResolverScore($Record, $Candidate) {
$nameMatches = ($Record.normalized_name -ne "" -and $Record.normalized_name -eq $Candidate.normalized_name)
$sizeMatches = ($Record.size_bytes -ge 0 -and $Record.size_bytes -eq $Candidate.size_bytes)
$sourceMatches = ($Record.source_id -ne "" -and $Candidate.search_text.Contains($Record.source_id))
if ($nameMatches -and $sizeMatches -and (Test-CloseTimestamp $Record $Candidate)) {
return 100
}
if ($sizeMatches -and $sourceMatches) {
return 80
}
if ($nameMatches) {
return 40
}
return 0
}
function Invoke-ResolverScoring($Records, $Candidates) {
$accepted = 0
$ambiguous = 0
$score100 = 0
$score80 = 0
$score40Only = 0
foreach ($record in @($Records)) {
$scores = @()
foreach ($candidate in @($Candidates)) {
$score = Get-ResolverScore $record $candidate
if ($score -gt 0) {
$scores += $score
}
}
if ($scores.Count -eq 0) {
continue
}
$top = ($scores | Sort-Object -Descending | Select-Object -First 1)
$topCount = @($scores | Where-Object { $_ -eq $top }).Count
if ($top -ge 80 -and $topCount -eq 1) {
$accepted++
if ($top -eq 100) { $score100++ }
elseif ($top -eq 80) { $score80++ }
}
elseif ($top -ge 80) {
$ambiguous++
}
elseif ($top -eq 40) {
$score40Only++
}
}
return [ordered]@{
records_checked = @($Records).Count
cache_candidates_checked = @($Candidates).Count
accepted_matches = $accepted
ambiguous_matches = $ambiguous
score_100_matches = $score100
score_80_matches = $score80
score_40_only_matches = $score40Only
used_fixture_records = $true
}
}
function Convert-CountObjectToHashtable($Value) {
$map = @{}
if ($null -eq $Value) { return $map }
@@ -522,7 +636,30 @@ elseif ($filenameOnlyMatches -gt 0 -or $sourceIDHashDirMatches -gt 0) {
$recommendedNextNode = "additional evidence request: collect file size or cache root metadata"
}
[ordered]@{
$resolverSummary = [ordered]@{
records_checked = [int]$dbProbe.record_count
cache_candidates_checked = [int]$cacheSummary["candidate_count"]
accepted_matches = 0
ambiguous_matches = 0
score_100_matches = 0
score_80_matches = 0
score_40_only_matches = $filenameOnlyMatches
used_fixture_records = $false
}
if (-not [string]::IsNullOrWhiteSpace($FixtureRecordsPath)) {
if ([string]::IsNullOrWhiteSpace($CacheRoot)) {
throw "FixtureRecordsPath requires CacheRoot"
}
$fixtureRecords = Read-FixtureFileRecords $FixtureRecordsPath
$cacheCandidates = Read-CacheCandidatesForScoring $CacheRoot
$resolverSummary = Invoke-ResolverScoring $fixtureRecords $cacheCandidates
$mappingValidated = ([int]$resolverSummary.accepted_matches -gt 0)
if ($mappingValidated) {
$recommendedNextNode = "R11 download preview resolver contract"
}
}
$summary = [ordered]@{
ok = $true
verification = "file_cache_mapping"
db_checked = $dbChecked
@@ -542,6 +679,14 @@ elseif ($filenameOnlyMatches -gt 0 -or $sourceIDHashDirMatches -gt 0) {
filename_size_match_count = $filenameSizeMatches
unique_filename_size_match_count = $uniqueFilenameSizeMatches
source_id_hash_dir_match_count = $sourceIDHashDirMatches
records_checked = [int]$resolverSummary.records_checked
cache_candidates_checked = [int]$resolverSummary.cache_candidates_checked
accepted_matches = [int]$resolverSummary.accepted_matches
ambiguous_matches = [int]$resolverSummary.ambiguous_matches
score_100_matches = [int]$resolverSummary.score_100_matches
score_80_matches = [int]$resolverSummary.score_80_matches
score_40_only_matches = [int]$resolverSummary.score_40_only_matches
used_fixture_records = [bool]$resolverSummary.used_fixture_records
mapping_validated = $mappingValidated
recommended_next_node = $recommendedNextNode
raw_paths_printed = $false
@@ -549,4 +694,14 @@ elseif ($filenameOnlyMatches -gt 0 -or $sourceIDHashDirMatches -gt 0) {
file_paths_returned = $false
raw_rows_returned = $false
message_body_values_printed = $false
} | ConvertTo-Json -Depth 8 -Compress
}
$json = $summary | ConvertTo-Json -Depth 8 -Compress
if (-not [string]::IsNullOrWhiteSpace($SanitizedOutputPath)) {
$parent = Split-Path -Parent $SanitizedOutputPath
if (-not [string]::IsNullOrWhiteSpace($parent)) {
New-Item -ItemType Directory -Force -Path $parent | Out-Null
}
Set-Content -LiteralPath $SanitizedOutputPath -Value $json -Encoding UTF8
}
$json