diff --git a/docs/current-status-card.md b/docs/current-status-card.md index 930c816..2fd9f62 100644 --- a/docs/current-status-card.md +++ b/docs/current-status-card.md @@ -59,7 +59,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme ## Current loop -Current loop: `R7 send-file preview tool complete`; next R8 send-file idempotency and audit hardening. +Current loop: `R10 receive-file download resolver v2 diagnostic complete`; next R11 receive-file download preview contract. Active continuous execution plan: `docs/superpowers/plans/2026-07-10-r6f-r14-continuous-execution-plan.md`. @@ -240,8 +240,8 @@ Continuous execution plan R6f-R14 is approved and execution has started: - Plan file: `docs/superpowers/plans/2026-07-10-r6f-r14-continuous-execution-plan.md`. - Scope: 15 ordered rounds covering send-message connector/gate hardening, send-file preview/idempotency/package, receive-file download preview, receive-message reconciliation, end-to-end business smoke, and release-candidate report. - Execution rule after approval: one round at a time, status-card update, verification, commit, push, then continue automatically until an evidence-only blocker requires user action. -- Last completed round: R9 send-file upload evidence package. -- Next active round: R10 receive-file download resolver v2 diagnostic. +- Last completed round: R10 receive-file download resolver v2 diagnostic. +- Next active round: R11 receive-file download preview contract. R6g send audit and idempotency replay diagnostics is complete: @@ -326,3 +326,12 @@ R9 send-file upload evidence package is complete: - Local verification generated and validated `runs\send-file-sandbox-gate-package.zip`; no production file upload connector was enabled. - Verification passed: git diff --check, scripts\verify-send-file-sandbox-gate-package.ps1, go test ./..., go build ./cmd/isphere-mcp, and scripts\verify-go-mcp.ps1. - Next node: R10 receive-file download resolver v2 diagnostic. + +R10 receive-file download resolver v2 diagnostic is complete: + +- Added fixture-backed scoring to `scripts\verify-file-cache-mapping.ps1` with `-FixtureRecordsPath`, `-CacheRoot`, and `-SanitizedOutputPath`. +- `scripts\test-verify-file-cache-mapping.ps1` now covers the legacy archive-list summary and the v2 scoring fixture. +- Fixture result: `records_checked=3`, `cache_candidates_checked=4`, `accepted_matches=2`, `ambiguous_matches=0`, `score_100_matches=1`, `score_80_matches=1`, and `score_40_only_matches=1`. +- Diagnostic output keeps `raw_paths_printed=false`, `file_contents_read=false`, and `file_paths_returned=false`. +- Decision: resolver logic is proven, but real production file download remains blocked until a real cache mapping package produces accepted non-ambiguous matches. +- Next node: R11 receive-file download preview contract. diff --git a/docs/source-discovery/2026-07-10-file-download-mapping-v2.md b/docs/source-discovery/2026-07-10-file-download-mapping-v2.md new file mode 100644 index 0000000..cdc412d --- /dev/null +++ b/docs/source-discovery/2026-07-10-file-download-mapping-v2.md @@ -0,0 +1,49 @@ +# File download mapping v2 diagnostic - 2026-07-10 + +## Purpose + +R10 checks whether received-file metadata can be mapped to an operator-local cache/download file strongly enough for a future `isphere_receive_files` download path. + +## What changed + +`scripts/verify-file-cache-mapping.ps1` now has a fixture-backed scoring mode: + +- `-FixtureRecordsPath`: JSON records with `file_name`, `size_bytes`, `source_id`, and `created_at`. +- `-CacheRoot`: operator-local cache root or fixture cache root. +- `-SanitizedOutputPath`: writes a compact JSON summary without raw paths or file contents. + +Scoring rules: + +| Rule | Score | Accepted | +| --- | ---: | --- | +| exact filename + exact size + close timestamp | 100 | yes, if unique | +| exact size + source id appears in cache name/ref | 80 | yes, if unique | +| filename only | 40 | no | + +## Fixture result + +`powershell -NoProfile -ExecutionPolicy Bypass -File scripts\test-verify-file-cache-mapping.ps1` produced: + +```json +{"ok":true,"archive_candidate_count":2,"records_checked":3,"cache_candidates_checked":4,"accepted_matches":2,"ambiguous_matches":0,"score_100_matches":1,"score_80_matches":1,"score_40_only_matches":1,"raw_paths_printed":false,"file_contents_read":false} +``` + +Interpretation: + +- The resolver algorithm can produce accepted matches when file metadata is strong. +- One weak filename-only case is intentionally not accepted. +- The diagnostic output stays committed-safe: no raw operator paths and no file contents. + +## Production decision + +This round proves the resolver logic, not a real iSphere cache mapping on this machine. The local machine still cannot log in to iSphere, and no real returned receive-file package in this repository state proves a stable cache/download mapping. + +Therefore: + +- `isphere_receive_files` list mode remains usable. +- Real file download/copy remains blocked until a real cache mapping package produces accepted, non-ambiguous matches. +- R11 should implement a download preview/blocked contract so digital employees get machine-readable status instead of a generic failure. + +## Next node + +R11: `Receive-file download preview contract`. diff --git a/docs/superpowers/plans/2026-07-10-r6f-r14-continuous-execution-plan.md b/docs/superpowers/plans/2026-07-10-r6f-r14-continuous-execution-plan.md index 71e69b5..7b1506a 100644 --- a/docs/superpowers/plans/2026-07-10-r6f-r14-continuous-execution-plan.md +++ b/docs/superpowers/plans/2026-07-10-r6f-r14-continuous-execution-plan.md @@ -709,7 +709,7 @@ git push gitea main - `-MsgLibDb` and `-SQLiteDll` accept copied DB/runtime paths when real copied DB is available; fixture tests do not require real DB files. - `-SanitizedOutputPath` writes committed-safe JSON summary to a caller-selected output file. -- [ ] **Step 1: Add fixture test for resolver scoring** +- [x] **Step 1: Add fixture test for resolver scoring** Fixture contains three DB-like file records and four cache-like file entries. Expected scoring: @@ -719,15 +719,15 @@ exact size + source id in name/ref => score 80 filename only => score 40 and not accepted ``` -- [ ] **Step 2: Implement scoring and sanitized output** +- [x] **Step 2: Implement scoring and sanitized output** Output fields: `records_checked`, `cache_candidates_checked`, `accepted_matches`, `ambiguous_matches`, `raw_paths_printed=false`, `file_contents_read=false`. -- [ ] **Step 3: Write decision note** +- [x] **Step 3: Write decision note** `docs/source-discovery/2026-07-10-file-download-mapping-v2.md` records whether accepted matches exist. If not, download remains blocked and UI/client download connector becomes the next route. -- [ ] **Step 4: Verify and commit** +- [x] **Step 4: Verify and commit** ```powershell git diff --check @@ -742,6 +742,14 @@ git push gitea main **Acceptance gate:** 文件下载要么得到可实现映射，要么得到明确阻断证据。 +**R10 Result:** + +- Added v2 fixture scoring to `scripts/verify-file-cache-mapping.ps1`. +- Updated `scripts/test-verify-file-cache-mapping.ps1` to prove exact filename/size/time gets score 100, source-id/size gets score 80, and filename-only gets score 40 without acceptance. +- Fixture verification produced `records_checked=3`, `cache_candidates_checked=4`, `accepted_matches=2`, `ambiguous_matches=0`, `score_100_matches=1`, `score_80_matches=1`, `score_40_only_matches=1`, `raw_paths_printed=false`, and `file_contents_read=false`. +- Created `docs/source-discovery/2026-07-10-file-download-mapping-v2.md`. +- Real production file download remains blocked because this proves resolver logic, not a real iSphere cache mapping from an online package. + --- ### R11: Receive-file download preview contract diff --git a/scripts/test-verify-file-cache-mapping.ps1 b/scripts/test-verify-file-cache-mapping.ps1 index 7164b33..26ccfb4 100644 --- a/scripts/test-verify-file-cache-mapping.ps1 +++ b/scripts/test-verify-file-cache-mapping.ps1 @@ -5,13 +5,22 @@ Set-StrictMode -Version Latest $repo = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path $script = Join-Path $repo "scripts\verify-file-cache-mapping.ps1" -$tempDir = Join-Path $repo ("runs\file-cache-mapping-test-" + [guid]::NewGuid().ToString("N")) +$tempDir = Join-Path ([System.IO.Path]::GetTempPath()) ("isphere-file-cache-mapping-test-" + [guid]::NewGuid().ToString("N")) $archiveList = Join-Path $tempDir "archive-list.txt" +$cacheRoot = Join-Path $tempDir "cache-root" +$recordsPath = Join-Path $tempDir "fixture-records.json" +$summaryPath = Join-Path $tempDir "sanitized-summary.json" function Assert-True([bool]$Condition, [string]$Message) { if (-not $Condition) { throw $Message } } +function Write-AsciiFile([string]$Path, [string]$Text) { + $parent = Split-Path -Parent $Path + New-Item -ItemType Directory -Force -Path $parent | Out-Null + [System.IO.File]::WriteAllBytes($Path, [System.Text.Encoding]::ASCII.GetBytes($Text)) +} + try { New-Item -ItemType Directory -Force -Path $tempDir | Out-Null @( @@ -20,21 +29,79 @@ try { "C:\redacted\other\ignored.txt" ) | Set-Content -LiteralPath $archiveList -Encoding UTF8 - $output = & powershell -NoProfile -ExecutionPolicy Bypass -File $script -ArchiveListPath $archiveList -SkipDb + $archiveOutput = & powershell -NoProfile -ExecutionPolicy Bypass -File $script -ArchiveListPath $archiveList -SkipDb if ($LASTEXITCODE -ne 0) { - throw "verify-file-cache-mapping.ps1 failed with exit code $LASTEXITCODE" + throw "archive-list verify-file-cache-mapping.ps1 failed with exit code $LASTEXITCODE" } - $json = $output | ConvertFrom-Json + $archiveJson = $archiveOutput | ConvertFrom-Json - Assert-True ($json.ok -eq $true) "ok should be true" - Assert-True ($json.cache_checked -eq $true) "cache should be checked" - Assert-True ($json.cache_candidate_count -eq 2) "cache candidate count should be 2" - Assert-True ($json.cache_extension_counts.".docx" -eq 1) "docx extension count should be 1" - Assert-True ($json.cache_extension_counts.".xlsx" -eq 1) "xlsx extension count should be 1" - Assert-True ($json.hash_dir_length_counts."32" -eq 2) "hash dir length count should be 2" - Assert-True ($json.raw_paths_printed -eq $false) "raw paths must not be printed" - Assert-True ($json.file_contents_read -eq $false) "file contents must not be read" - Assert-True ($json.file_paths_returned -eq $false) "file paths must not be returned" + Assert-True ($archiveJson.ok -eq $true) "archive ok should be true" + Assert-True ($archiveJson.cache_checked -eq $true) "archive cache should be checked" + Assert-True ($archiveJson.cache_candidate_count -eq 2) "archive cache candidate count should be 2" + Assert-True ($archiveJson.cache_extension_counts.".docx" -eq 1) "docx extension count should be 1" + Assert-True ($archiveJson.cache_extension_counts.".xlsx" -eq 1) "xlsx extension count should be 1" + Assert-True ($archiveJson.hash_dir_length_counts."32" -eq 2) "hash dir length count should be 2" + Assert-True ($archiveJson.raw_paths_printed -eq $false) "archive raw paths must not be printed" + Assert-True ($archiveJson.file_contents_read -eq $false) "archive file contents must not be read" + Assert-True ($archiveJson.file_paths_returned -eq $false) "archive file paths must not be returned" + + $reportPath = Join-Path $cacheRoot "stable\report.docx" + $payloadPath = Join-Path $cacheRoot "source-src-abc-123\payload.bin" + $loosePath = Join-Path $cacheRoot "loose\loose.txt" + $noisePath = Join-Path $cacheRoot "noise\noise.tmp" + Write-AsciiFile $reportPath "hello" + Write-AsciiFile $payloadPath "payload" + Write-AsciiFile $loosePath "abc" + Write-AsciiFile $noisePath "noise" + (Get-Item -LiteralPath $reportPath).LastWriteTimeUtc = [datetime]"2026-07-10T04:00:10Z" + (Get-Item -LiteralPath $payloadPath).LastWriteTimeUtc = [datetime]"2026-07-10T05:00:00Z" + (Get-Item -LiteralPath $loosePath).LastWriteTimeUtc = [datetime]"2026-07-10T06:00:00Z" + (Get-Item -LiteralPath $noisePath).LastWriteTimeUtc = [datetime]"2026-07-10T07:00:00Z" + + @( + [ordered]@{ file_name = "report.docx"; size_bytes = 5; source_id = "report-source"; created_at = "2026-07-10T04:00:00Z" }, + [ordered]@{ file_name = "receipt.pdf"; size_bytes = 7; source_id = "src-abc-123"; created_at = "2026-07-10T04:10:00Z" }, + [ordered]@{ file_name = "loose.txt"; size_bytes = 999; source_id = "loose-source"; created_at = "2026-07-10T04:20:00Z" } + ) | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $recordsPath -Encoding UTF8 + + $fixtureOutput = & powershell -NoProfile -ExecutionPolicy Bypass -File $script -SkipDb -CacheRoot $cacheRoot -FixtureRecordsPath $recordsPath -SanitizedOutputPath $summaryPath + if ($LASTEXITCODE -ne 0) { + throw "fixture verify-file-cache-mapping.ps1 failed with exit code $LASTEXITCODE" + } + $fixtureJson = $fixtureOutput | ConvertFrom-Json + + Assert-True ($fixtureJson.ok -eq $true) "fixture ok should be true" + Assert-True ($fixtureJson.records_checked -eq 3) "records_checked should be 3" + Assert-True ($fixtureJson.cache_candidates_checked -eq 4) "cache_candidates_checked should be 4" + Assert-True ($fixtureJson.accepted_matches -eq 2) "accepted_matches should be 2" + Assert-True ($fixtureJson.ambiguous_matches -eq 0) "ambiguous_matches should be 0" + Assert-True ($fixtureJson.score_100_matches -eq 1) "score_100_matches should be 1" + Assert-True ($fixtureJson.score_80_matches -eq 1) "score_80_matches should be 1" + Assert-True ($fixtureJson.score_40_only_matches -eq 1) "score_40_only_matches should be 1" + Assert-True ($fixtureJson.used_fixture_records -eq $true) "used_fixture_records should be true" + Assert-True ($fixtureJson.mapping_validated -eq $true) "fixture mapping should be validated" + Assert-True ($fixtureJson.raw_paths_printed -eq $false) "fixture raw paths must not be printed" + Assert-True ($fixtureJson.file_contents_read -eq $false) "fixture file contents must not be read" + Assert-True (Test-Path -LiteralPath $summaryPath) "sanitized summary should be written" + + $summaryText = Get-Content -LiteralPath $summaryPath -Raw + Assert-True (-not $summaryText.Contains($cacheRoot)) "sanitized summary must not contain raw cache root" + Assert-True (-not $summaryText.Contains("hello")) "sanitized summary must not contain file content" + Assert-True (-not $summaryText.Contains("payload")) "sanitized summary must not contain file content" + + [ordered]@{ + ok = $true + archive_candidate_count = $archiveJson.cache_candidate_count + records_checked = $fixtureJson.records_checked + cache_candidates_checked = $fixtureJson.cache_candidates_checked + accepted_matches = $fixtureJson.accepted_matches + ambiguous_matches = $fixtureJson.ambiguous_matches + score_100_matches = $fixtureJson.score_100_matches + score_80_matches = $fixtureJson.score_80_matches + score_40_only_matches = $fixtureJson.score_40_only_matches + raw_paths_printed = $fixtureJson.raw_paths_printed + file_contents_read = $fixtureJson.file_contents_read + } | ConvertTo-Json -Depth 4 -Compress } finally { if (Test-Path -LiteralPath $tempDir) { diff --git a/scripts/verify-file-cache-mapping.ps1 b/scripts/verify-file-cache-mapping.ps1 index f129d31..505804f 100644 --- a/scripts/verify-file-cache-mapping.ps1 +++ b/scripts/verify-file-cache-mapping.ps1 @@ -4,6 +4,8 @@ param( [string]$SQLiteDllPath = $env:ISPHERE_MSGLIB_SQLITE_DLL, [string]$DbPath = $env:ISPHERE_MSGLIB_DB, [string]$Password = "123", + [string]$FixtureRecordsPath = "", + [string]$SanitizedOutputPath = "", [switch]$SkipDb ) @@ -116,6 +118,118 @@ function Read-CacheSummaryFromRoot([string]$Path) { return $summary } +function ConvertTo-SafeFileRecord($Value) { + $createdAt = $null + if (-not [string]::IsNullOrWhiteSpace([string]$Value.created_at)) { + [datetime]$parsed = [datetime]::MinValue + if ([datetime]::TryParse([string]$Value.created_at, [ref]$parsed)) { + $createdAt = $parsed.ToUniversalTime() + } + } + return [pscustomobject]@{ + normalized_name = Normalize-FileName ([string]$Value.file_name) + size_bytes = if ($null -ne $Value.size_bytes) { [int64]$Value.size_bytes } else { -1 } + source_id = ([string]$Value.source_id).Trim().ToLowerInvariant() + created_at_utc = $createdAt + } +} + +function Read-FixtureFileRecords([string]$Path) { + if (-not (Test-Path -LiteralPath $Path)) { + throw "FixtureRecordsPath not found" + } + $json = Get-Content -LiteralPath $Path -Raw | ConvertFrom-Json + $safeRecords = @() + foreach ($record in $json) { + $safeRecords += (ConvertTo-SafeFileRecord $record) + } + return $safeRecords +} + +function Read-CacheCandidatesForScoring([string]$Path) { + if (-not (Test-Path -LiteralPath $Path)) { + throw "CacheRoot not found" + } + $candidates = @() + foreach ($file in Get-ChildItem -LiteralPath $Path -File -Recurse) { + $searchText = (($file.Name + " " + $file.DirectoryName) -replace '[\\/]', ' ').ToLowerInvariant() + $candidates += [pscustomobject]@{ + normalized_name = Normalize-FileName $file.Name + size_bytes = [int64]$file.Length + last_write_utc = $file.LastWriteTimeUtc + search_text = $searchText + } + } + return $candidates +} + +function Test-CloseTimestamp($Record, $Candidate) { + if ($null -eq $Record.created_at_utc -or $null -eq $Candidate.last_write_utc) { + return $false + } + $delta = [Math]::Abs((New-TimeSpan -Start $Record.created_at_utc -End $Candidate.last_write_utc).TotalSeconds) + return $delta -le 600 +} + +function Get-ResolverScore($Record, $Candidate) { + $nameMatches = ($Record.normalized_name -ne "" -and $Record.normalized_name -eq $Candidate.normalized_name) + $sizeMatches = ($Record.size_bytes -ge 0 -and $Record.size_bytes -eq $Candidate.size_bytes) + $sourceMatches = ($Record.source_id -ne "" -and $Candidate.search_text.Contains($Record.source_id)) + if ($nameMatches -and $sizeMatches -and (Test-CloseTimestamp $Record $Candidate)) { + return 100 + } + if ($sizeMatches -and $sourceMatches) { + return 80 + } + if ($nameMatches) { + return 40 + } + return 0 +} + +function Invoke-ResolverScoring($Records, $Candidates) { + $accepted = 0 + $ambiguous = 0 + $score100 = 0 + $score80 = 0 + $score40Only = 0 + foreach ($record in @($Records)) { + $scores = @() + foreach ($candidate in @($Candidates)) { + $score = Get-ResolverScore $record $candidate + if ($score -gt 0) { + $scores += $score + } + } + if ($scores.Count -eq 0) { + continue + } + $top = ($scores | Sort-Object -Descending | Select-Object -First 1) + $topCount = @($scores | Where-Object { $_ -eq $top }).Count + if ($top -ge 80 -and $topCount -eq 1) { + $accepted++ + if ($top -eq 100) { $score100++ } + elseif ($top -eq 80) { $score80++ } + } + elseif ($top -ge 80) { + $ambiguous++ + } + elseif ($top -eq 40) { + $score40Only++ + } + } + return [ordered]@{ + records_checked = @($Records).Count + cache_candidates_checked = @($Candidates).Count + accepted_matches = $accepted + ambiguous_matches = $ambiguous + score_100_matches = $score100 + score_80_matches = $score80 + score_40_only_matches = $score40Only + used_fixture_records = $true + } +} + function Convert-CountObjectToHashtable($Value) { $map = @{} if ($null -eq $Value) { return $map } @@ -522,7 +636,30 @@ elseif ($filenameOnlyMatches -gt 0 -or $sourceIDHashDirMatches -gt 0) { $recommendedNextNode = "additional evidence request: collect file size or cache root metadata" } -[ordered]@{ +$resolverSummary = [ordered]@{ + records_checked = [int]$dbProbe.record_count + cache_candidates_checked = [int]$cacheSummary["candidate_count"] + accepted_matches = 0 + ambiguous_matches = 0 + score_100_matches = 0 + score_80_matches = 0 + score_40_only_matches = $filenameOnlyMatches + used_fixture_records = $false +} +if (-not [string]::IsNullOrWhiteSpace($FixtureRecordsPath)) { + if ([string]::IsNullOrWhiteSpace($CacheRoot)) { + throw "FixtureRecordsPath requires CacheRoot" + } + $fixtureRecords = Read-FixtureFileRecords $FixtureRecordsPath + $cacheCandidates = Read-CacheCandidatesForScoring $CacheRoot + $resolverSummary = Invoke-ResolverScoring $fixtureRecords $cacheCandidates + $mappingValidated = ([int]$resolverSummary.accepted_matches -gt 0) + if ($mappingValidated) { + $recommendedNextNode = "R11 download preview resolver contract" + } +} + +$summary = [ordered]@{ ok = $true verification = "file_cache_mapping" db_checked = $dbChecked @@ -542,6 +679,14 @@ elseif ($filenameOnlyMatches -gt 0 -or $sourceIDHashDirMatches -gt 0) { filename_size_match_count = $filenameSizeMatches unique_filename_size_match_count = $uniqueFilenameSizeMatches source_id_hash_dir_match_count = $sourceIDHashDirMatches + records_checked = [int]$resolverSummary.records_checked + cache_candidates_checked = [int]$resolverSummary.cache_candidates_checked + accepted_matches = [int]$resolverSummary.accepted_matches + ambiguous_matches = [int]$resolverSummary.ambiguous_matches + score_100_matches = [int]$resolverSummary.score_100_matches + score_80_matches = [int]$resolverSummary.score_80_matches + score_40_only_matches = [int]$resolverSummary.score_40_only_matches + used_fixture_records = [bool]$resolverSummary.used_fixture_records mapping_validated = $mappingValidated recommended_next_node = $recommendedNextNode raw_paths_printed = $false @@ -549,4 +694,14 @@ elseif ($filenameOnlyMatches -gt 0 -or $sourceIDHashDirMatches -gt 0) { file_paths_returned = $false raw_rows_returned = $false message_body_values_printed = $false -} | ConvertTo-Json -Depth 8 -Compress +} + +$json = $summary | ConvertTo-Json -Depth 8 -Compress +if (-not [string]::IsNullOrWhiteSpace($SanitizedOutputPath)) { + $parent = Split-Path -Parent $SanitizedOutputPath + if (-not [string]::IsNullOrWhiteSpace($parent)) { + New-Item -ItemType Directory -Force -Path $parent | Out-Null + } + Set-Content -LiteralPath $SanitizedOutputPath -Value $json -Encoding UTF8 +} +$json