diff --git a/docs/current-status-card.md b/docs/current-status-card.md index 24c837e..1ddbf32 100644 --- a/docs/current-status-card.md +++ b/docs/current-status-card.md @@ -27,7 +27,7 @@ Remote base before local roadmap commits: `b2d839e Merge branch 'codex/n15-repor - `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, eight-tool MCP surface, default empty-source receive/contact/group/file calls, synthetic configured-file receive/contact/group/file smoke, and synthetic configured-directory receive/contact/group/file smoke. - `isphere_receive_messages` now returns the contract-facing `ok`, `conversation`, `messages`, `next_cursor`, and `audit` envelope while retaining legacy parser-native message fields for compatibility. - `isphere_receive_messages` accepts the read-contract argument set for the current local-readonly path: `conversation_id`, `query`, `since`, `limit`, `include_attachment_metadata`, `source_preference`, `preview`, and empty `cursor`. -- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown because C23 found no validated mapping source yet. +- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown because C23/C24 found no validated DB read path yet. - `isphere_receive_files` list mode accepts and validates the safe file-list contract args for the current local-readonly path while keeping download blocked. - N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures. - N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only. @@ -55,7 +55,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme ## Current loop -Current loop: `Stage C / Loop C24 - MsgLib.db readability/wrapper path precheck`. +Current loop: `Stage C / Loop C25 - static .NET DB wrapper analysis`. Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`. @@ -91,10 +91,11 @@ Current loop output so far: 21. C21: `isphere_search_contacts` and `isphere_search_groups` now accept and validate safe search-contract args, while rejecting unsupported source/cursor cases. 22. C22: `isphere_receive_files` list mode now accepts and validates safe contract args, rejects unsupported source/cursor/output/download cases, and keeps download blocked. 23. C23: display-name enrichment was blocked because current committed evidence and ignored metadata do not validate a JID-to-display-name/group-title source; `MsgLib.db` remains the most likely next source but inspected copies are not directly readable as SQLite. +24. C24: `MsgLib.db` readability precheck found high-entropy non-SQLite files with a consistent wrapper/encryption signature; wrapper candidates include `HYHC.IMPP.DAL.dll`, `Utilities.Lib.SQLiteInteraction*.dll`, `Utilities.Lib.DBModel.dll`, and `RepairDatabase.exe`. ## Next business mainline -1. Run Stage C Loop C24: precheck why `MsgLib.db` is not directly readable as SQLite and identify the safest wrapper/decryption/read path before attempting display-name enrichment. +1. Run Stage C Loop C25: statically inspect the .NET DB wrapper assemblies to map table/model names and connection-string/password/decryption flow before any copied-DB read attempt. 2. Leave `isphere_send_message` blocked until bridge/API, UIA action-chain, or protocol connector evidence is validated. 3. Leave actual file download/cache mapping as a later connector node until a message-to-cache hash or client download source is validated. 4. Keep UIA selector/report work as fallback support, not as the primary roadmap. diff --git a/docs/source-discovery/2026-07-10-msglib-readability-precheck.md b/docs/source-discovery/2026-07-10-msglib-readability-precheck.md new file mode 100644 index 0000000..38c37df --- /dev/null +++ b/docs/source-discovery/2026-07-10-msglib-readability-precheck.md @@ -0,0 +1,85 @@ +# MsgLib.db readability and wrapper-path precheck + +Date: 2026-07-10 + +## Question + +Why are the copied `MsgLib.db` files not directly readable as SQLite, and what is the safest next path for extracting contact/group display names and richer message metadata? + +## Evidence checked + +- Existing ignored metadata: + - `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/schema-inspection-summary.json` + - `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/archive-list.txt` +- New ignored C24 metadata: + - `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/c24-msglib-format-precheck.json` + - `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/c24-wrapper-string-precheck.json` +- Read-only extracted wrapper candidates under ignored: + - `runs/offline-evidence-intake/zyl-qqfile-20260709/extracted/c24-wrapper-candidates/` + +No raw DB bytes, decrypted messages, or personal values are committed in this note. + +## Findings + +1. The three inspected `MsgLib.db` copies do not start with SQLite magic and contain no SQLite header within the first 1 MiB sample. +2. The inspected DB samples have high entropy and the same non-SQLite prefix shape across iSphere and IMPP account copies. This strongly suggests a consistent encrypted/wrapped DB format rather than a random corrupt copy. +3. The archive contains standard SQLite libraries, DB abstraction libraries, and a repair tool next to `MsgLib.db`: + - `System.Data.SQLite.dll` + - `Utilities.Lib.DBSQLite.dll` + - `Utilities.Lib.SQLiteInteraction.dll` + - `Utilities.Lib.SQLiteInteractionBase.dll` + - `Utilities.Lib.DBBase.dll` + - `Utilities.Lib.DBModel.dll` + - `HYHC.IMPP.DAL.dll` + - `RepairDatabase.exe` +4. String-only static scanning of wrapper candidates shows that `HYHC.IMPP.DAL.dll` and `RepairDatabase.exe` reference message/group DB behavior and candidate table/method names, including: + - `tblMsgGroupPersonMsg` + - `ImportOldDataBase` + - `GetGroupMessaeByID` + - `GetGroupRecentMsg` + - `UpdateGroupName` + - `SelectMsgGroupName` + - `SelectRecentGroupName` + - `SaveGroupInfomation` + - `groupJid` + - `set_MemberJid` + - `set_MsgGroupName` + - `set_MemberName` + - `get_naturalname` +5. `Utilities.Lib.SQLiteInteraction.dll` exposes `SQLiteConnectionStringBuilder` plus `get_Password` and `set_Password`. +6. `Utilities.Lib.DBModel.dll` exposes configuration concepts such as `IConStrDecryption`, `PasswordPropertyName`, and `DecryptionPropertyName`. +7. `smcrypto-*` and `BouncyCastle.Crypto.dll` are present, but C24 does not prove they are the actual `MsgLib.db` wrapper. They are supporting crypto candidates only. + +## Decision + +Do not treat `MsgLib.db` as ordinary SQLite. + +The best current hypothesis is: + +> `MsgLib.db` is an encrypted or wrapped SQLite database opened through the client DB abstraction layer, likely involving `Utilities.Lib.SQLiteInteraction*`, `Utilities.Lib.DBModel`, and `HYHC.IMPP.DAL`. + +The next safe implementation path is not direct DB reads. It is static wrapper analysis first: + +1. Inspect .NET metadata/IL for the DB wrapper assemblies. +2. Identify how connection strings/password/decryption are supplied. +3. Identify table names and model classes for message, contact, group, member, and display-name fields. +4. Only after that, attempt a read-only schema extraction on copied DB files under ignored paths. + +## Next slice + +C25 should perform a read-only static .NET wrapper analysis over the extracted candidate assemblies. It should produce table/model/source mappings only, not execute the client and not write to the original DB. + +Suggested C25 focus: + +- `HYHC.IMPP.DAL.dll` +- `Utilities.Lib.SQLiteInteraction.dll` +- `Utilities.Lib.DBModel.dll` +- `Utilities.Lib.DBSQLite.dll` +- `RepairDatabase.exe` + +Expected output: + +- candidate table names; +- candidate model/property names for contact/group/message metadata; +- likely connection-string/password/decryption flow; +- decision whether C26 can attempt a copied-DB read-only schema extraction. diff --git a/docs/source-discovery/capability-source-matrix.md b/docs/source-discovery/capability-source-matrix.md index b37f605..3103470 100644 --- a/docs/source-discovery/capability-source-matrix.md +++ b/docs/source-discovery/capability-source-matrix.md @@ -18,8 +18,8 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md` | MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice | | --- | --- | --- | --- | --- | --- | | `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; wrapped `MsgLib.db` after DB wrapper is solved | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence` | C20 supports and validates the current safe receive-message contract args; default source remains empty when unset | apply safe contract args to contact/group search | -| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | decrypted roster/contact stanzas from `Smark.SendReceive`; `MsgLib.db` contact/conversation tables after DB wrapper is solved; UIA helper source if display names are missing | `docs/source-discovery/2026-07-10-display-name-source-precheck.md` | C23 blocked display-name enrichment: no validated JID-to-display-name source yet; JID-only display/account fallback remains honest | precheck `MsgLib.db` readability/wrapper path | -| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | `MsgLib.db` group/conversation tables after DB wrapper is solved; UIA helper source if group display names are missing | `docs/source-discovery/2026-07-10-display-name-source-precheck.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C23 blocked display-name/member enrichment: no validated group-title/member source yet; JID-only display fallback remains honest | precheck `MsgLib.db` readability/wrapper path | +| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | decrypted roster/contact stanzas from `Smark.SendReceive`; `MsgLib.db` contact/conversation tables after DB wrapper is solved; UIA helper source if display names are missing | `docs/source-discovery/2026-07-10-msglib-readability-precheck.md` | C24 confirms `MsgLib.db` is not ordinary SQLite and likely needs client DB wrapper/static .NET analysis before display-name extraction | static wrapper analysis for table/model/password flow | +| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | `MsgLib.db` group/conversation tables after DB wrapper is solved; UIA helper source if group display names are missing | `docs/source-discovery/2026-07-10-msglib-readability-precheck.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C24 finds group table/model candidates in wrapper strings but no validated copied-DB read path yet; JID-only display fallback remains honest | static wrapper analysis for table/model/password flow | | `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence | | `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration | | `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated | diff --git a/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md b/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md index db70168..4927552 100644 --- a/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md +++ b/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md @@ -1925,15 +1925,15 @@ Evidence precheck result: - If not identifiable, record the blocker and shift to another business-read hardening node. - Do not add write/send/file-download behavior. -- [ ] **Step 1: Inspect safe DB metadata** +- [x] **Step 1: Inspect safe DB metadata** Use file headers, existing summaries, and committed notes only. Avoid committing raw bytes. -- [ ] **Step 2: Record likely wrapper/read path** +- [x] **Step 2: Record likely wrapper/read path** Write a source-discovery note with the decision and next implementation slice. -- [ ] **Step 3: Verify and commit docs** +- [x] **Step 3: Verify and commit docs** Run: @@ -1947,6 +1947,72 @@ Expected: all exit 0. Commit exact C24 paths only and rewrite the next loop from --- +## Loop C24 Result + +Evidence precheck result: + +- Three inspected `MsgLib.db` copies do not start with SQLite magic and contain no SQLite header in the first 1 MiB sample. +- The files have high entropy and share the same non-SQLite prefix shape across iSphere and IMPP copies, so the likely issue is an encrypted/wrapped DB format rather than a random corrupt copy. +- Archive metadata shows SQLite and DB wrapper assemblies next to the client: + - `System.Data.SQLite.dll` + - `Utilities.Lib.DBSQLite.dll` + - `Utilities.Lib.SQLiteInteraction.dll` + - `Utilities.Lib.SQLiteInteractionBase.dll` + - `Utilities.Lib.DBBase.dll` + - `Utilities.Lib.DBModel.dll` + - `HYHC.IMPP.DAL.dll` + - `RepairDatabase.exe` +- Ignored string-only scans of extracted wrapper candidates show relevant method/table/model hints including `tblMsgGroupPersonMsg`, `ImportOldDataBase`, `GetGroupRecentMsg`, `UpdateGroupName`, `SelectMsgGroupName`, `SelectRecentGroupName`, `groupJid`, `set_MemberJid`, `set_MsgGroupName`, `set_MemberName`, and `get_naturalname`. +- `Utilities.Lib.SQLiteInteraction.dll` exposes SQLite connection builder plus password accessors; `Utilities.Lib.DBModel.dll` exposes connection-string decryption/password configuration concepts. +- Decision: do not read `MsgLib.db` as ordinary SQLite. The next safe route is static .NET wrapper analysis before any copied-DB read attempt. + +--- + +## Loop C25: Static .NET DB wrapper analysis + +**Goal:** Statically inspect the extracted DB wrapper assemblies to map candidate table/model names and connection-string/password/decryption flow for future read-only `MsgLib.db` access. + +**Planned files:** +- Modify: `docs/source-discovery/capability-source-matrix.md` +- Modify: `docs/current-status-card.md` +- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md` +- Optionally create a focused source-discovery note under `docs/source-discovery/`. +- Optionally create ignored metadata under `runs/`; do not commit binaries, raw DB bytes, decrypted content, or personal values. + +**Planned behavior:** +- Inspect .NET assembly metadata/IL read-only for: + - `HYHC.IMPP.DAL.dll` + - `Utilities.Lib.SQLiteInteraction.dll` + - `Utilities.Lib.DBModel.dll` + - `Utilities.Lib.DBSQLite.dll` + - `RepairDatabase.exe` +- Extract only table/model/method/property names and connection/decryption flow summaries. +- Decide whether C26 can attempt copied-DB read-only schema extraction or must stay blocked. +- Do not execute client binaries or modify DB files. +- Do not add write/send/file-download behavior. + +- [ ] **Step 1: Generate static wrapper metadata** + +Use read-only metadata/IL tools or string extraction. Keep output under ignored metadata. + +- [ ] **Step 2: Write source-discovery conclusion** + +Record likely tables/models and the safest next read path. + +- [ ] **Step 3: Verify and commit docs** + +Run: + +```powershell +git diff --check +go test ./... +powershell -NoProfile -ExecutionPolicy Bypass -File scripts/verify-go-mcp.ps1 +``` + +Expected: all exit 0. Commit exact C25 paths only and rewrite the next loop from actual results. + +--- + ## Self-Review - Spec coverage: the plan follows the user's cyclic requirement: plan first, implement one loop at a time, update the next loop after each result, and ask only on blockers.