feat: add file cache mapping diagnostic
This commit is contained in:
zhaoyilun
@@ -56,7 +56,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme
|
||||
|
||||
## Current loop
|
||||
|
||||
Current loop: `Business Roadmap / R3b - File cache mapping diagnostic`.
|
||||
Current loop: `Business Roadmap / R5 - Send-message connector discovery`.
|
||||
|
||||
Plan file: `docs/superpowers/plans/2026-07-10-core-business-capabilities-roadmap.md`.
|
||||
|
||||
@@ -139,3 +139,10 @@ R3 receive-file download mapping precheck is complete:
|
||||
- Created `docs/source-discovery/2026-07-10-file-download-mapping-precheck.md`.
|
||||
- Decision: do not implement download mode yet; current evidence supports list mode and DB attachment metadata, but not a validated cache/download mapping.
|
||||
- Next node: R3b file cache mapping diagnostic with sanitized counts only.
|
||||
|
||||
R3b file cache mapping diagnostic is complete:
|
||||
|
||||
- Added `scripts/verify-file-cache-mapping.ps1` and a fixture test script.
|
||||
- Real sanitized diagnostic found 166 MsgLib received-file records and 11 cache/archive candidates, but 0 filename/source-id/cache matches.
|
||||
- Decision: R4 receive-file download implementation is blocked until better cache evidence or a UI/client download connector is available.
|
||||
- Next node: R5 send-message connector discovery.
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
# File Cache Mapping Diagnostic
|
||||
|
||||
Date: 2026-07-10
|
||||
|
||||
## Goal
|
||||
|
||||
Run the R3b local-only diagnostic to determine whether copied MsgLib received-file metadata can be mapped to the available ignored cache/archive listing without printing file paths, raw rows, message bodies, or attachment contents.
|
||||
|
||||
## Diagnostic command shape
|
||||
|
||||
The diagnostic script is:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-file-cache-mapping.ps1 \
|
||||
-ArchiveListPath <ignored-archive-list>
|
||||
```
|
||||
|
||||
The script can also accept `ISPHERE_MSGLIB_SQLITE_DLL` and `ISPHERE_MSGLIB_DB` from the environment. The committed note intentionally omits local DB, DLL, and archive-list path values.
|
||||
|
||||
## Sanitized result
|
||||
|
||||
Safe summary from the available copied DB plus ignored archive-list metadata:
|
||||
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"verification": "file_cache_mapping",
|
||||
"db_checked": true,
|
||||
"db_table_available": true,
|
||||
"db_record_count": 166,
|
||||
"db_filename_present_count": 166,
|
||||
"db_size_present_count": 166,
|
||||
"db_source_id_present_count": 166,
|
||||
"db_receive_msg_guid_present_count": 166,
|
||||
"cache_checked": true,
|
||||
"cache_candidate_count": 11,
|
||||
"hash_dir_length_counts": {"32": 11},
|
||||
"filename_only_match_count": 0,
|
||||
"filename_size_match_count": 0,
|
||||
"unique_filename_size_match_count": 0,
|
||||
"source_id_hash_dir_match_count": 0,
|
||||
"mapping_validated": false,
|
||||
"recommended_next_node": "additional evidence request",
|
||||
"raw_paths_printed": false,
|
||||
"file_contents_read": false,
|
||||
"file_paths_returned": false,
|
||||
"raw_rows_returned": false,
|
||||
"message_body_values_printed": false
|
||||
}
|
||||
```
|
||||
|
||||
Extension counts are allowed as coarse metadata. The diagnostic normalizes abnormal extensions to `.other` so file-name fragments are not printed.
|
||||
|
||||
## Decision
|
||||
|
||||
Do not implement R4 receive-file download mode from the current evidence.
|
||||
|
||||
Reason:
|
||||
|
||||
- MsgLib received-file metadata exists and is internally complete enough for metadata analysis: filename, size, source id, and receive message id are present for 166 inspected rows.
|
||||
- The available cache/archive listing has 11 hash-shape cache candidates.
|
||||
- The diagnostic found 0 filename-only matches, 0 filename+size matches, and 0 source-id-to-hash-dir matches.
|
||||
- Therefore no unique or repeatable cache-file resolver can be claimed.
|
||||
|
||||
## Next route
|
||||
|
||||
File download remains blocked until one of these is available:
|
||||
|
||||
1. A better operator-local cache root or fresh archive-list that actually contains the same received files as `TD_ReceiveFileRecord`.
|
||||
2. A known recent received-file sample where the operator can confirm the message/file metadata and resulting local cache object, with committed output still limited to counts and booleans.
|
||||
3. A UI/client download connector path that can trigger download without relying on cache guessing.
|
||||
|
||||
The broader roadmap should skip R4 for now and continue with R5 send-message connector discovery.
|
||||
@@ -20,7 +20,7 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`
|
||||
| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | copied `MsgLib.db` message tables through x86 read-only sidecar `list_messages` selected explicitly by `source_preference="msglib_readonly"`; decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md`; `docs/source-discovery/2026-07-10-db-backed-receive-source-design.md`; `docs/source-discovery/2026-07-10-receive-message-reconciliation-precheck.md` | C20 supports and validates the current safe receive-message contract args; C32 wires optional MsgLib display metadata into receive-message `sender_name` and `conversation.display_name`; C33 proves this path through a real copied-DB MCP smoke; C35 maps MsgLib message tables; C36 adds metadata-only `message_sources`; C37 defines bounded `list_messages` design; C38 implements sidecar/Go-wrapper `list_messages` with sanitized copied-DB verification; C39 adds a Go adapter from `ListMessages` to the receive-message domain model; C40 adds tool-level explicit `msglib_readonly` source selection; C41 wires env-configured MsgLib receive source and optional copied-DB smoke; R1 precheck keeps `auto` PacketReader/log-backed until a future reconciliation helper is implemented | future receive reconciliation helper after R2/R3 |
|
||||
| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; `internal/isphere/contacts_test.go`; `internal/tools/isphere_contacts_test.go` | C34 documents optional MsgLib contact display enrichment; R2 adds deterministic exact-match-first ranking, case-insensitive de-duplication across log/MsgLib candidates, and preserves `source`/`raw_ref` | core contact search complete; optional richer fields later |
|
||||
| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; `internal/isphere/groups_test.go`; `internal/tools/isphere_groups_test.go`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C34 documents optional MsgLib group display enrichment; R2 adds deterministic exact-match-first ranking, case-insensitive de-duplication across log/MsgLib candidates, and preserves `source`/`raw_ref` | core group search complete; optional member/owner hierarchy later |
|
||||
| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | MsgLib `TD_ReceiveFileRecord` safe metadata through explicit copied-DB path; decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck`; `docs/source-discovery/2026-07-10-file-download-mapping-precheck.md` | C22 verifies safe file-list contract args; R3 confirms list metadata is available but no validated message/file-record-to-cache-file mapping exists; R4 remains blocked | R3b file cache mapping diagnostic |
|
||||
| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | MsgLib `TD_ReceiveFileRecord` safe metadata through explicit copied-DB path; decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck`; `docs/source-discovery/2026-07-10-file-download-mapping-precheck.md`; `docs/source-discovery/2026-07-10-file-cache-mapping-diagnostic.md` | C22 verifies safe file-list contract args; R3 confirms list metadata is available; R3b diagnostic finds 0 current cache/archive matches, so R4 remains blocked | blocked pending better cache evidence or UI/client download connector |
|
||||
| `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | R5 send-message connector discovery |
|
||||
| `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | R7 send-file connector discovery |
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@
|
||||
| Search groups | Registered MCP tool; deterministic exact-match ranking and case-insensitive de-duplication across groupchat/conference candidates and optional copied-DB MsgLib display enrichment. | Optional member count, owner refs, and nested/group hierarchy remain future enrichment, not a core search blocker. | complete for core search; optional enrichment later |
|
||||
| Receive messages | PacketReader source works; copied `MsgLib.db` explicit receive works with `source_preference="msglib_readonly"`; R1 precheck says default should remain PacketReader until a future reconciliation helper is implemented. | Future reconciliation helper, source confidence scoring, pagination policy. | future receive helper after R2/R3 |
|
||||
| Send messages | Contract exists; no validated write connector. | Need connector discovery, preview-only path, idempotency and audit before production send. | R5 then R6 |
|
||||
| Receive files | List mode exists from message-derived file metadata; R3 precheck confirms cache/download mapping is still unvalidated. | Need sanitized cache mapping diagnostic before output-dir download behavior. | R3b then R4 if validated |
|
||||
| Receive files | List mode exists from message-derived file metadata; R3/R3b confirm DB file metadata exists but current cache/archive evidence does not map to local files. | R4 download is blocked until better cache evidence or a UI/client download connector is available. | blocked; continue R5 |
|
||||
| Send files | Contract exists; no validated upload/send connector. | Depends on send-message connector plus upload/file-transfer evidence. | R7 then R8 |
|
||||
|
||||
## Node Sequence
|
||||
@@ -259,6 +259,15 @@ powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-go-mcp.ps1
|
||||
**Stop condition:**
|
||||
- If no unique/repeatable cache mapping exists, do not implement R4 download mode.
|
||||
|
||||
**R3b Result:**
|
||||
|
||||
- Added `scripts/verify-file-cache-mapping.ps1` and `scripts/test-verify-file-cache-mapping.ps1`.
|
||||
- Created `docs/source-discovery/2026-07-10-file-cache-mapping-diagnostic.md`.
|
||||
- Real sanitized diagnostic checked 166 `TD_ReceiveFileRecord` rows and 11 ignored cache/archive candidates.
|
||||
- Match counts were all zero: filename-only, filename+size, unique filename+size, and source-id-to-hash-dir.
|
||||
- Safety booleans stayed false for raw paths, file contents, file paths returned, raw rows, and message body printing.
|
||||
- Decision: R4 remains blocked; continue the broader roadmap at R5 send-message connector discovery.
|
||||
|
||||
---
|
||||
|
||||
### R4: Receive-file download implementation
|
||||
|
||||
Reference in New Issue
Block a user