feat: add file cache mapping diagnostic

2026-07-10 09:52:34 +08:00
parent ea38346ce3
commit 5578f0bda7
6 changed files with 687 additions and 3 deletions
--- a/docs/current-status-card.md
+++ b/docs/current-status-card.md
@@ -56,7 +56,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme

 ## Current loop

-Current loop: `Business Roadmap / R3b - File cache mapping diagnostic`.
+Current loop: `Business Roadmap / R5 - Send-message connector discovery`.

 Plan file: `docs/superpowers/plans/2026-07-10-core-business-capabilities-roadmap.md`.

@@ -139,3 +139,10 @@ R3 receive-file download mapping precheck is complete:
 - Created `docs/source-discovery/2026-07-10-file-download-mapping-precheck.md`.
 - Decision: do not implement download mode yet; current evidence supports list mode and DB attachment metadata, but not a validated cache/download mapping.
 - Next node: R3b file cache mapping diagnostic with sanitized counts only.
+
+R3b file cache mapping diagnostic is complete:
+
+- Added `scripts/verify-file-cache-mapping.ps1` and a fixture test script.
+- Real sanitized diagnostic found 166 MsgLib received-file records and 11 cache/archive candidates, but 0 filename/source-id/cache matches.
+- Decision: R4 receive-file download implementation is blocked until better cache evidence or a UI/client download connector is available.
+- Next node: R5 send-message connector discovery.
--- a/docs/source-discovery/2026-07-10-file-cache-mapping-diagnostic.md
+++ b/docs/source-discovery/2026-07-10-file-cache-mapping-diagnostic.md
@@ -0,0 +1,73 @@
+# File Cache Mapping Diagnostic
+
+Date: 2026-07-10
+
+## Goal
+
+Run the R3b local-only diagnostic to determine whether copied MsgLib received-file metadata can be mapped to the available ignored cache/archive listing without printing file paths, raw rows, message bodies, or attachment contents.
+
+## Diagnostic command shape
+
+The diagnostic script is:
+
+```powershell
+powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-file-cache-mapping.ps1 \
+  -ArchiveListPath <ignored-archive-list>
+```
+
+The script can also accept `ISPHERE_MSGLIB_SQLITE_DLL` and `ISPHERE_MSGLIB_DB` from the environment. The committed note intentionally omits local DB, DLL, and archive-list path values.
+
+## Sanitized result
+
+Safe summary from the available copied DB plus ignored archive-list metadata:
+
+```json
+{
+  "ok": true,
+  "verification": "file_cache_mapping",
+  "db_checked": true,
+  "db_table_available": true,
+  "db_record_count": 166,
+  "db_filename_present_count": 166,
+  "db_size_present_count": 166,
+  "db_source_id_present_count": 166,
+  "db_receive_msg_guid_present_count": 166,
+  "cache_checked": true,
+  "cache_candidate_count": 11,
+  "hash_dir_length_counts": {"32": 11},
+  "filename_only_match_count": 0,
+  "filename_size_match_count": 0,
+  "unique_filename_size_match_count": 0,
+  "source_id_hash_dir_match_count": 0,
+  "mapping_validated": false,
+  "recommended_next_node": "additional evidence request",
+  "raw_paths_printed": false,
+  "file_contents_read": false,
+  "file_paths_returned": false,
+  "raw_rows_returned": false,
+  "message_body_values_printed": false
+}
+```
+
+Extension counts are allowed as coarse metadata. The diagnostic normalizes abnormal extensions to `.other` so file-name fragments are not printed.
+
+## Decision
+
+Do not implement R4 receive-file download mode from the current evidence.
+
+Reason:
+
+- MsgLib received-file metadata exists and is internally complete enough for metadata analysis: filename, size, source id, and receive message id are present for 166 inspected rows.
+- The available cache/archive listing has 11 hash-shape cache candidates.
+- The diagnostic found 0 filename-only matches, 0 filename+size matches, and 0 source-id-to-hash-dir matches.
+- Therefore no unique or repeatable cache-file resolver can be claimed.
+
+## Next route
+
+File download remains blocked until one of these is available:
+
+1. A better operator-local cache root or fresh archive-list that actually contains the same received files as `TD_ReceiveFileRecord`.
+2. A known recent received-file sample where the operator can confirm the message/file metadata and resulting local cache object, with committed output still limited to counts and booleans.
+3. A UI/client download connector path that can trigger download without relying on cache guessing.
+
+The broader roadmap should skip R4 for now and continue with R5 send-message connector discovery.
--- a/docs/source-discovery/capability-source-matrix.md
+++ b/docs/source-discovery/capability-source-matrix.md
@@ -20,7 +20,7 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`
 | `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | copied `MsgLib.db` message tables through x86 read-only sidecar `list_messages` selected explicitly by `source_preference="msglib_readonly"`; decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md`; `docs/source-discovery/2026-07-10-db-backed-receive-source-design.md`; `docs/source-discovery/2026-07-10-receive-message-reconciliation-precheck.md` | C20 supports and validates the current safe receive-message contract args; C32 wires optional MsgLib display metadata into receive-message `sender_name` and `conversation.display_name`; C33 proves this path through a real copied-DB MCP smoke; C35 maps MsgLib message tables; C36 adds metadata-only `message_sources`; C37 defines bounded `list_messages` design; C38 implements sidecar/Go-wrapper `list_messages` with sanitized copied-DB verification; C39 adds a Go adapter from `ListMessages` to the receive-message domain model; C40 adds tool-level explicit `msglib_readonly` source selection; C41 wires env-configured MsgLib receive source and optional copied-DB smoke; R1 precheck keeps `auto` PacketReader/log-backed until a future reconciliation helper is implemented | future receive reconciliation helper after R2/R3 |
 | `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; `internal/isphere/contacts_test.go`; `internal/tools/isphere_contacts_test.go` | C34 documents optional MsgLib contact display enrichment; R2 adds deterministic exact-match-first ranking, case-insensitive de-duplication across log/MsgLib candidates, and preserves `source`/`raw_ref` | core contact search complete; optional richer fields later |
 | `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; `internal/isphere/groups_test.go`; `internal/tools/isphere_groups_test.go`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C34 documents optional MsgLib group display enrichment; R2 adds deterministic exact-match-first ranking, case-insensitive de-duplication across log/MsgLib candidates, and preserves `source`/`raw_ref` | core group search complete; optional member/owner hierarchy later |
-| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | MsgLib `TD_ReceiveFileRecord` safe metadata through explicit copied-DB path; decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck`; `docs/source-discovery/2026-07-10-file-download-mapping-precheck.md` | C22 verifies safe file-list contract args; R3 confirms list metadata is available but no validated message/file-record-to-cache-file mapping exists; R4 remains blocked | R3b file cache mapping diagnostic |
+| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | MsgLib `TD_ReceiveFileRecord` safe metadata through explicit copied-DB path; decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck`; `docs/source-discovery/2026-07-10-file-download-mapping-precheck.md`; `docs/source-discovery/2026-07-10-file-cache-mapping-diagnostic.md` | C22 verifies safe file-list contract args; R3 confirms list metadata is available; R3b diagnostic finds 0 current cache/archive matches, so R4 remains blocked | blocked pending better cache evidence or UI/client download connector |
 | `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | R5 send-message connector discovery |
 | `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | R7 send-file connector discovery |

--- a/docs/superpowers/plans/2026-07-10-core-business-capabilities-roadmap.md
+++ b/docs/superpowers/plans/2026-07-10-core-business-capabilities-roadmap.md
@@ -29,7 +29,7 @@
 | Search groups | Registered MCP tool; deterministic exact-match ranking and case-insensitive de-duplication across groupchat/conference candidates and optional copied-DB MsgLib display enrichment. | Optional member count, owner refs, and nested/group hierarchy remain future enrichment, not a core search blocker. | complete for core search; optional enrichment later |
 | Receive messages | PacketReader source works; copied `MsgLib.db` explicit receive works with `source_preference="msglib_readonly"`; R1 precheck says default should remain PacketReader until a future reconciliation helper is implemented. | Future reconciliation helper, source confidence scoring, pagination policy. | future receive helper after R2/R3 |
 | Send messages | Contract exists; no validated write connector. | Need connector discovery, preview-only path, idempotency and audit before production send. | R5 then R6 |
-| Receive files | List mode exists from message-derived file metadata; R3 precheck confirms cache/download mapping is still unvalidated. | Need sanitized cache mapping diagnostic before output-dir download behavior. | R3b then R4 if validated |
+| Receive files | List mode exists from message-derived file metadata; R3/R3b confirm DB file metadata exists but current cache/archive evidence does not map to local files. | R4 download is blocked until better cache evidence or a UI/client download connector is available. | blocked; continue R5 |
 | Send files | Contract exists; no validated upload/send connector. | Depends on send-message connector plus upload/file-transfer evidence. | R7 then R8 |

 ## Node Sequence
@@ -259,6 +259,15 @@ powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-go-mcp.ps1
 **Stop condition:**
 - If no unique/repeatable cache mapping exists, do not implement R4 download mode.

+**R3b Result:**
+
+- Added `scripts/verify-file-cache-mapping.ps1` and `scripts/test-verify-file-cache-mapping.ps1`.
+- Created `docs/source-discovery/2026-07-10-file-cache-mapping-diagnostic.md`.
+- Real sanitized diagnostic checked 166 `TD_ReceiveFileRecord` rows and 11 ignored cache/archive candidates.
+- Match counts were all zero: filename-only, filename+size, unique filename+size, and source-id-to-hash-dir.
+- Safety booleans stayed false for raw paths, file contents, file paths returned, raw rows, and message body printing.
+- Decision: R4 remains blocked; continue the broader roadmap at R5 send-message connector discovery.
+
 ---

 ### R4: Receive-file download implementation
--- a/scripts/test-verify-file-cache-mapping.ps1
+++ b/scripts/test-verify-file-cache-mapping.ps1
@@ -0,0 +1,43 @@
+param()
+
+$ErrorActionPreference = "Stop"
+Set-StrictMode -Version Latest
+
+$repo = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path
+$script = Join-Path $repo "scripts\verify-file-cache-mapping.ps1"
+$tempDir = Join-Path $repo ("runs\file-cache-mapping-test-" + [guid]::NewGuid().ToString("N"))
+$archiveList = Join-Path $tempDir "archive-list.txt"
+
+function Assert-True([bool]$Condition, [string]$Message) {
+    if (-not $Condition) { throw $Message }
+}
+
+try {
+    New-Item -ItemType Directory -Force -Path $tempDir | Out-Null
+    @(
+        "C:\redacted\importal\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\redacted-report.docx",
+        "C:\redacted\importal\bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\redacted-sheet.xlsx",
+        "C:\redacted\other\ignored.txt"
+    ) | Set-Content -LiteralPath $archiveList -Encoding UTF8
+
+    $output = & powershell -NoProfile -ExecutionPolicy Bypass -File $script -ArchiveListPath $archiveList -SkipDb
+    if ($LASTEXITCODE -ne 0) {
+        throw "verify-file-cache-mapping.ps1 failed with exit code $LASTEXITCODE"
+    }
+    $json = $output | ConvertFrom-Json
+
+    Assert-True ($json.ok -eq $true) "ok should be true"
+    Assert-True ($json.cache_checked -eq $true) "cache should be checked"
+    Assert-True ($json.cache_candidate_count -eq 2) "cache candidate count should be 2"
+    Assert-True ($json.cache_extension_counts.".docx" -eq 1) "docx extension count should be 1"
+    Assert-True ($json.cache_extension_counts.".xlsx" -eq 1) "xlsx extension count should be 1"
+    Assert-True ($json.hash_dir_length_counts."32" -eq 2) "hash dir length count should be 2"
+    Assert-True ($json.raw_paths_printed -eq $false) "raw paths must not be printed"
+    Assert-True ($json.file_contents_read -eq $false) "file contents must not be read"
+    Assert-True ($json.file_paths_returned -eq $false) "file paths must not be returned"
+}
+finally {
+    if (Test-Path -LiteralPath $tempDir) {
+        Remove-Item -LiteralPath $tempDir -Recurse -Force
+    }
+}
--- a/scripts/verify-file-cache-mapping.ps1
+++ b/scripts/verify-file-cache-mapping.ps1
@@ -0,0 +1,552 @@
+param(
+    [string]$ArchiveListPath = "",
+    [string]$CacheRoot = "",
+    [string]$SQLiteDllPath = $env:ISPHERE_MSGLIB_SQLITE_DLL,
+    [string]$DbPath = $env:ISPHERE_MSGLIB_DB,
+    [string]$Password = "123",
+    [switch]$SkipDb
+)
+
+$ErrorActionPreference = "Stop"
+Set-StrictMode -Version Latest
+
+$repo = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")).Path
+
+function Add-Count([hashtable]$Map, [string]$Key, [int]$Increment = 1) {
+    if ([string]::IsNullOrWhiteSpace($Key)) { return }
+    if ($Map.ContainsKey($Key)) {
+        $Map[$Key] = [int]$Map[$Key] + $Increment
+    }
+    else {
+        $Map[$Key] = $Increment
+    }
+}
+
+function Get-Sha256Text([string]$Value) {
+    $sha = [System.Security.Cryptography.SHA256]::Create()
+    try {
+        $bytes = [System.Text.Encoding]::UTF8.GetBytes($Value)
+        $hash = $sha.ComputeHash($bytes)
+        return (($hash | ForEach-Object { $_.ToString("x2") }) -join "")
+    }
+    finally {
+        $sha.Dispose()
+    }
+}
+
+function Normalize-FileName([string]$Value) {
+    if ([string]::IsNullOrWhiteSpace($Value)) { return "" }
+    $leaf = ($Value -split '[\\/]')[-1]
+    return $leaf.Trim().ToLowerInvariant()
+}
+
+function Normalize-Extension([string]$Value) {
+    if ([string]::IsNullOrWhiteSpace($Value)) { return "" }
+    $extension = $Value.Trim().ToLowerInvariant()
+    if ($extension -match '^\.[a-z0-9]{1,8}$') { return $extension }
+    return ".other"
+}
+
+function New-EmptyCacheSummary {
+    return [ordered]@{
+        checked = $false
+        candidate_count = 0
+        extension_counts = @{}
+        hash_dir_length_counts = @{}
+        hash_dir_value_hash_counts = @{}
+        filename_hash_counts = @{}
+        filename_size_hash_counts = @{}
+        has_size_data = $false
+    }
+}
+
+function Add-CacheCandidate($Summary, [string]$Name, [Nullable[Int64]]$Size, [string]$HashDir) {
+    $normalizedName = Normalize-FileName $Name
+    if ($normalizedName -eq "") { return }
+    $extension = Normalize-Extension ([System.IO.Path]::GetExtension($normalizedName))
+    if ($extension -ne "") {
+        Add-Count $Summary.extension_counts $extension
+    }
+    if (-not [string]::IsNullOrWhiteSpace($HashDir)) {
+        Add-Count $Summary.hash_dir_length_counts ([string]$HashDir.Length)
+        Add-Count $Summary.hash_dir_value_hash_counts (Get-Sha256Text $HashDir.ToLowerInvariant())
+    }
+    Add-Count $Summary.filename_hash_counts (Get-Sha256Text $normalizedName)
+    if ($null -ne $Size -and $Size.Value -ge 0) {
+        $Summary["has_size_data"] = $true
+        Add-Count $Summary.filename_size_hash_counts (Get-Sha256Text ($normalizedName + "|" + $Size.Value))
+    }
+    $Summary["candidate_count"] = [int]$Summary["candidate_count"] + 1
+}
+
+function Get-HashDirFromPathText([string]$Value) {
+    $match = [regex]::Match($Value, '(?i)(?:^|[\\/])importal[\\/]([0-9a-f]{32,})(?:[\\/])')
+    if ($match.Success) {
+        return $match.Groups[1].Value.ToLowerInvariant()
+    }
+    return ""
+}
+
+function Read-CacheSummaryFromArchiveList([string]$Path) {
+    if (-not (Test-Path -LiteralPath $Path)) {
+        throw "ArchiveListPath not found"
+    }
+    $summary = New-EmptyCacheSummary
+    $summary["checked"] = $true
+    foreach ($line in Get-Content -LiteralPath $Path) {
+        $text = [string]$line
+        $hashDir = Get-HashDirFromPathText $text
+        if ($hashDir -eq "") { continue }
+        Add-CacheCandidate $summary $text $null $hashDir
+    }
+    return $summary
+}
+
+function Read-CacheSummaryFromRoot([string]$Path) {
+    if (-not (Test-Path -LiteralPath $Path)) {
+        throw "CacheRoot not found"
+    }
+    $summary = New-EmptyCacheSummary
+    $summary["checked"] = $true
+    foreach ($file in Get-ChildItem -LiteralPath $Path -File -Recurse) {
+        $hashDir = Get-HashDirFromPathText $file.FullName
+        if ($hashDir -eq "") { continue }
+        Add-CacheCandidate $summary $file.Name ([Int64]$file.Length) $hashDir
+    }
+    return $summary
+}
+
+function Convert-CountObjectToHashtable($Value) {
+    $map = @{}
+    if ($null -eq $Value) { return $map }
+    if ($Value -is [System.Collections.IDictionary]) {
+        foreach ($key in $Value.Keys) {
+            $map[[string]$key] = [int]$Value[$key]
+        }
+        return $map
+    }
+    foreach ($property in $Value.PSObject.Properties) {
+        $map[$property.Name] = [int]$property.Value
+    }
+    return $map
+}
+
+function Count-Intersections([hashtable]$Left, [hashtable]$Right) {
+    $count = 0
+    foreach ($key in $Left.Keys) {
+        if ($Right.ContainsKey($key)) {
+            $count += [Math]::Min([int]$Left[$key], [int]$Right[$key])
+        }
+    }
+    return $count
+}
+
+function Count-UniqueIntersections([hashtable]$Left, [hashtable]$Right) {
+    $count = 0
+    foreach ($key in $Left.Keys) {
+        if ($Right.ContainsKey($key) -and [int]$Left[$key] -eq 1 -and [int]$Right[$key] -eq 1) {
+            $count++
+        }
+    }
+    return $count
+}
+
+function Invoke-ReceiveFileRecordProbe([string]$SQLiteDll, [string]$Database, [string]$DbPassword) {
+    if (-not (Test-Path -LiteralPath $SQLiteDll)) {
+        throw "SQLite DLL not found"
+    }
+    if (-not (Test-Path -LiteralPath $Database)) {
+        throw "MsgLib DB not found"
+    }
+
+    $tempDir = Join-Path $repo ("runs\file-cache-mapping-db-probe-" + [guid]::NewGuid().ToString("N"))
+    $source = Join-Path $tempDir "ReceiveFileRecordProbe.cs"
+    $exe = Join-Path $tempDir "ReceiveFileRecordProbe.exe"
+    $sqliteCopy = Join-Path $tempDir "System.Data.SQLite.dll"
+    $csc = "$env:WINDIR\Microsoft.NET\Framework\v4.0.30319\csc.exe"
+    if (-not (Test-Path -LiteralPath $csc)) {
+        throw "32-bit .NET Framework csc.exe v4.0.30319 not found"
+    }
+
+    try {
+        New-Item -ItemType Directory -Force -Path $tempDir | Out-Null
+        Copy-Item -LiteralPath $SQLiteDll -Destination $sqliteCopy -Force
+        @'
+using System;
+using System.Collections.Generic;
+using System.Data.Common;
+using System.IO;
+using System.Reflection;
+using System.Security.Cryptography;
+using System.Text;
+using System.Web.Script.Serialization;
+
+internal static class ReceiveFileRecordProbe
+{
+    private static int Main(string[] args)
+    {
+        try
+        {
+            string sqliteDllPath = args[0];
+            string dbPath = args[1];
+            string password = args.Length > 2 ? args[2] : "123";
+            AppDomain.CurrentDomain.AssemblyResolve += delegate(object sender, ResolveEventArgs eventArgs)
+            {
+                if (eventArgs.Name.StartsWith("System.Data.SQLite", StringComparison.OrdinalIgnoreCase))
+                {
+                    return Assembly.LoadFrom(sqliteDllPath);
+                }
+                return null;
+            };
+
+            Assembly sqliteAssembly = Assembly.LoadFrom(sqliteDllPath);
+            Type factoryType = sqliteAssembly.GetType("System.Data.SQLite.SQLiteFactory", true);
+            FieldInfo instanceField = factoryType.GetField("Instance", BindingFlags.Public | BindingFlags.Static);
+            DbProviderFactory factory = (DbProviderFactory)instanceField.GetValue(null);
+
+            var result = NewResult();
+            using (DbConnection connection = factory.CreateConnection())
+            {
+                connection.ConnectionString = "Data Source=" + dbPath + ";Version=3;Read Only=True;Password=" + password + ";";
+                connection.Open();
+                ExecuteNonQuery(connection, "PRAGMA query_only=ON");
+                if (!TableExists(connection, "TD_ReceiveFileRecord"))
+                {
+                    PrintJson(result);
+                    return 0;
+                }
+                result["table_available"] = true;
+                HashSet<string> columns = ReadColumns(connection, "TD_ReceiveFileRecord");
+                string msgColumn = ExistingColumn(columns, "ReceiveMsgGuid");
+                string nameColumn = ExistingColumn(columns, "FileName");
+                string sizeColumn = ExistingColumn(columns, "FileSize", "Size");
+                string sourceColumn = ExistingColumn(columns, "SourceID", "FileGuid", "FileId");
+                if (nameColumn == "")
+                {
+                    PrintJson(result);
+                    return 0;
+                }
+
+                string sql = "select " +
+                    SelectColumn(msgColumn, "receive_msg_guid") + ", " +
+                    SelectColumn(nameColumn, "file_name") + ", " +
+                    SelectColumn(sizeColumn, "size_bytes") + ", " +
+                    SelectColumn(sourceColumn, "source_id") +
+                    " from " + QuoteIdent("TD_ReceiveFileRecord") + " limit 100000";
+
+                using (DbCommand command = connection.CreateCommand())
+                {
+                    command.CommandText = sql;
+                    using (DbDataReader reader = command.ExecuteReader())
+                    {
+                        while (reader.Read())
+                        {
+                            result["record_count"] = (int)result["record_count"] + 1;
+                            string receiveMsgGuid = ReaderString(reader, "receive_msg_guid");
+                            string fileName = SafeFileName(ReaderString(reader, "file_name"));
+                            long size = ReaderInt64(reader, "size_bytes");
+                            string sourceId = ReaderString(reader, "source_id").Trim().ToLowerInvariant();
+
+                            if (receiveMsgGuid.Trim() != "") result["receive_msg_guid_present_count"] = (int)result["receive_msg_guid_present_count"] + 1;
+                            if (fileName.Trim() != "") result["filename_present_count"] = (int)result["filename_present_count"] + 1;
+                            if (size > 0) result["size_present_count"] = (int)result["size_present_count"] + 1;
+                            if (sourceId != "") result["source_id_present_count"] = (int)result["source_id_present_count"] + 1;
+
+                            string normalizedName = fileName.Trim().ToLowerInvariant();
+                            if (normalizedName != "")
+                            {
+                                AddCount((Dictionary<string, object>)result["extension_counts"], SafeExtension(Path.GetExtension(normalizedName)));
+                                AddCount((Dictionary<string, object>)result["filename_hash_counts"], Sha256(normalizedName));
+                                if (size > 0) AddCount((Dictionary<string, object>)result["filename_size_hash_counts"], Sha256(normalizedName + "|" + size.ToString()));
+                            }
+                            if (sourceId != "")
+                            {
+                                AddCount((Dictionary<string, object>)result["source_id_hash_counts"], Sha256(sourceId));
+                                if (IsHexLike(sourceId)) result["source_id_hash_shape_count"] = (int)result["source_id_hash_shape_count"] + 1;
+                            }
+                        }
+                    }
+                }
+            }
+            PrintJson(result);
+            return 0;
+        }
+        catch (Exception ex)
+        {
+            Console.Error.WriteLine(ex.GetType().Name + ": " + ex.Message);
+            return 1;
+        }
+    }
+
+    private static Dictionary<string, object> NewResult()
+    {
+        return new Dictionary<string, object>
+        {
+            { "table_available", false },
+            { "record_count", 0 },
+            { "filename_present_count", 0 },
+            { "size_present_count", 0 },
+            { "source_id_present_count", 0 },
+            { "receive_msg_guid_present_count", 0 },
+            { "source_id_hash_shape_count", 0 },
+            { "extension_counts", new Dictionary<string, object>() },
+            { "filename_hash_counts", new Dictionary<string, object>() },
+            { "filename_size_hash_counts", new Dictionary<string, object>() },
+            { "source_id_hash_counts", new Dictionary<string, object>() }
+        };
+    }
+
+    private static void PrintJson(Dictionary<string, object> value)
+    {
+        Console.WriteLine(new JavaScriptSerializer().Serialize(value));
+    }
+
+    private static void AddCount(Dictionary<string, object> map, string key)
+    {
+        if (string.IsNullOrWhiteSpace(key)) return;
+        if (map.ContainsKey(key)) map[key] = (int)map[key] + 1;
+        else map[key] = 1;
+    }
+
+    private static bool TableExists(DbConnection connection, string table)
+    {
+        using (DbCommand command = connection.CreateCommand())
+        {
+            command.CommandText = "select count(*) from sqlite_master where type='table' and name=@name";
+            DbParameter parameter = command.CreateParameter();
+            parameter.ParameterName = "@name";
+            parameter.Value = table;
+            command.Parameters.Add(parameter);
+            return Convert.ToInt32(command.ExecuteScalar()) > 0;
+        }
+    }
+
+    private static HashSet<string> ReadColumns(DbConnection connection, string table)
+    {
+        var columns = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
+        using (DbCommand command = connection.CreateCommand())
+        {
+            command.CommandText = "pragma table_info(" + QuoteIdent(table) + ")";
+            using (DbDataReader reader = command.ExecuteReader())
+            {
+                while (reader.Read()) columns.Add(Convert.ToString(reader["name"]));
+            }
+        }
+        return columns;
+    }
+
+    private static string ExistingColumn(HashSet<string> columns, params string[] names)
+    {
+        foreach (string name in names)
+        {
+            if (!string.IsNullOrWhiteSpace(name) && columns.Contains(name)) return name;
+        }
+        return "";
+    }
+
+    private static string SelectColumn(string column, string alias)
+    {
+        if (string.IsNullOrWhiteSpace(column)) return "'' as " + QuoteIdent(alias);
+        return QuoteIdent(column) + " as " + QuoteIdent(alias);
+    }
+
+    private static string QuoteIdent(string value)
+    {
+        return "\"" + value.Replace("\"", "\"\"") + "\"";
+    }
+
+    private static string ReaderString(DbDataReader reader, string column)
+    {
+        object value = reader[column];
+        if (value == null || value == DBNull.Value) return "";
+        return Convert.ToString(value);
+    }
+
+    private static long ReaderInt64(DbDataReader reader, string column)
+    {
+        object value = reader[column];
+        if (value == null || value == DBNull.Value) return 0;
+        long parsed;
+        if (long.TryParse(Convert.ToString(value), out parsed)) return parsed;
+        return 0;
+    }
+
+    private static string SafeFileName(string value)
+    {
+        if (string.IsNullOrWhiteSpace(value)) return "";
+        try { return Path.GetFileName(value); }
+        catch
+        {
+            int slash = Math.Max(value.LastIndexOf('/'), value.LastIndexOf('\\'));
+            if (slash >= 0 && slash + 1 < value.Length) return value.Substring(slash + 1);
+            return value;
+        }
+    }
+
+    private static string SafeExtension(string value)
+    {
+        if (string.IsNullOrWhiteSpace(value)) return "";
+        string extension = value.Trim().ToLowerInvariant();
+        if (extension.Length < 2 || extension.Length > 9) return ".other";
+        if (extension[0] != '.') return ".other";
+        for (int i = 1; i < extension.Length; i++)
+        {
+            char c = extension[i];
+            bool ok = (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9');
+            if (!ok) return ".other";
+        }
+        return extension;
+    }
+
+    private static bool IsHexLike(string value)
+    {
+        if (value.Length < 16) return false;
+        foreach (char c in value)
+        {
+            bool ok = (c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F');
+            if (!ok) return false;
+        }
+        return true;
+    }
+
+    private static string Sha256(string value)
+    {
+        using (SHA256 sha = SHA256.Create())
+        {
+            byte[] hash = sha.ComputeHash(Encoding.UTF8.GetBytes(value));
+            var builder = new StringBuilder(hash.Length * 2);
+            foreach (byte b in hash) builder.Append(b.ToString("x2"));
+            return builder.ToString();
+        }
+    }
+
+    private static void ExecuteNonQuery(DbConnection connection, string sql)
+    {
+        using (DbCommand command = connection.CreateCommand())
+        {
+            command.CommandText = sql;
+            command.ExecuteNonQuery();
+        }
+    }
+}
+'@ | Set-Content -LiteralPath $source -Encoding UTF8
+
+        & $csc `
+            /nologo `
+            /target:exe `
+            /platform:x86 `
+            /optimize+ `
+            /warn:0 `
+            /out:$exe `
+            /reference:System.dll `
+            /reference:System.Core.dll `
+            /reference:System.Data.dll `
+            /reference:System.Web.Extensions.dll `
+            /reference:$sqliteCopy `
+            $source | Out-Null
+        if ($LASTEXITCODE -ne 0) {
+            throw "ReceiveFileRecordProbe compilation failed"
+        }
+        @'
+<?xml version="1.0" encoding="utf-8" ?>
+<configuration>
+  <startup useLegacyV2RuntimeActivationPolicy="true">
+    <supportedRuntime version="v4.0" />
+    <supportedRuntime version="v2.0.50727" />
+  </startup>
+</configuration>
+'@ | Set-Content -LiteralPath ($exe + ".config") -Encoding UTF8
+
+        $output = & $exe $sqliteCopy $Database $DbPassword
+        if ($LASTEXITCODE -ne 0) {
+            throw "ReceiveFileRecordProbe failed"
+        }
+        return $output | ConvertFrom-Json
+    }
+    finally {
+        if (Test-Path -LiteralPath $tempDir) {
+            Remove-Item -LiteralPath $tempDir -Recurse -Force
+        }
+    }
+}
+
+$cacheSummary = New-EmptyCacheSummary
+if (-not [string]::IsNullOrWhiteSpace($ArchiveListPath)) {
+    $cacheSummary = Read-CacheSummaryFromArchiveList $ArchiveListPath
+}
+elseif (-not [string]::IsNullOrWhiteSpace($CacheRoot)) {
+    $cacheSummary = Read-CacheSummaryFromRoot $CacheRoot
+}
+
+$dbChecked = $false
+$dbProbe = [ordered]@{
+    table_available = $false
+    record_count = 0
+    filename_present_count = 0
+    size_present_count = 0
+    source_id_present_count = 0
+    receive_msg_guid_present_count = 0
+    source_id_hash_shape_count = 0
+    extension_counts = @{}
+    filename_hash_counts = @{}
+    filename_size_hash_counts = @{}
+    source_id_hash_counts = @{}
+}
+
+if (-not $SkipDb) {
+    if ([string]::IsNullOrWhiteSpace($SQLiteDllPath) -or [string]::IsNullOrWhiteSpace($DbPath)) {
+        throw "MsgLib DB probe requires SQLiteDllPath and DbPath, or pass -SkipDb"
+    }
+    $dbProbe = Invoke-ReceiveFileRecordProbe $SQLiteDllPath $DbPath $Password
+    $dbChecked = $true
+}
+
+$dbFilenameHashes = Convert-CountObjectToHashtable $dbProbe.filename_hash_counts
+$dbFilenameSizeHashes = Convert-CountObjectToHashtable $dbProbe.filename_size_hash_counts
+$dbSourceIDHashes = Convert-CountObjectToHashtable $dbProbe.source_id_hash_counts
+$cacheFilenameHashes = [hashtable]$cacheSummary["filename_hash_counts"]
+$cacheFilenameSizeHashes = [hashtable]$cacheSummary["filename_size_hash_counts"]
+$cacheHashDirValueHashes = [hashtable]$cacheSummary["hash_dir_value_hash_counts"]
+
+$filenameOnlyMatches = Count-Intersections $dbFilenameHashes $cacheFilenameHashes
+$filenameSizeMatches = Count-Intersections $dbFilenameSizeHashes $cacheFilenameSizeHashes
+$uniqueFilenameSizeMatches = Count-UniqueIntersections $dbFilenameSizeHashes $cacheFilenameSizeHashes
+$sourceIDHashDirMatches = Count-Intersections $dbSourceIDHashes $cacheHashDirValueHashes
+
+$mappingValidated = ($uniqueFilenameSizeMatches -gt 0 -and [bool]$cacheSummary["has_size_data"])
+$recommendedNextNode = "additional evidence request"
+if ($mappingValidated) {
+    $recommendedNextNode = "R4 cache-file resolver candidate"
+}
+elseif ($filenameOnlyMatches -gt 0 -or $sourceIDHashDirMatches -gt 0) {
+    $recommendedNextNode = "additional evidence request: collect file size or cache root metadata"
+}
+
+[ordered]@{
+    ok = $true
+    verification = "file_cache_mapping"
+    db_checked = $dbChecked
+    db_table_available = [bool]$dbProbe.table_available
+    db_record_count = [int]$dbProbe.record_count
+    db_filename_present_count = [int]$dbProbe.filename_present_count
+    db_size_present_count = [int]$dbProbe.size_present_count
+    db_source_id_present_count = [int]$dbProbe.source_id_present_count
+    db_receive_msg_guid_present_count = [int]$dbProbe.receive_msg_guid_present_count
+    db_source_id_hash_shape_count = [int]$dbProbe.source_id_hash_shape_count
+    db_extension_counts = $dbProbe.extension_counts
+    cache_checked = [bool]$cacheSummary["checked"]
+    cache_candidate_count = [int]$cacheSummary["candidate_count"]
+    cache_extension_counts = $cacheSummary["extension_counts"]
+    hash_dir_length_counts = $cacheSummary["hash_dir_length_counts"]
+    filename_only_match_count = $filenameOnlyMatches
+    filename_size_match_count = $filenameSizeMatches
+    unique_filename_size_match_count = $uniqueFilenameSizeMatches
+    source_id_hash_dir_match_count = $sourceIDHashDirMatches
+    mapping_validated = $mappingValidated
+    recommended_next_node = $recommendedNextNode
+    raw_paths_printed = $false
+    file_contents_read = $false
+    file_paths_returned = $false
+    raw_rows_returned = $false
+    message_body_values_printed = $false
+} | ConvertTo-Json -Depth 8 -Compress