docs: identify msglib wrapper analysis path

This commit is contained in:
zhaoyilun
2026-07-10 02:17:23 +08:00
parent 3f052125ad
commit 7ced6e45f6
4 changed files with 160 additions and 8 deletions

View File

@@ -27,7 +27,7 @@ Remote base before local roadmap commits: `b2d839e Merge branch 'codex/n15-repor
- `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, eight-tool MCP surface, default empty-source receive/contact/group/file calls, synthetic configured-file receive/contact/group/file smoke, and synthetic configured-directory receive/contact/group/file smoke. - `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, eight-tool MCP surface, default empty-source receive/contact/group/file calls, synthetic configured-file receive/contact/group/file smoke, and synthetic configured-directory receive/contact/group/file smoke.
- `isphere_receive_messages` now returns the contract-facing `ok`, `conversation`, `messages`, `next_cursor`, and `audit` envelope while retaining legacy parser-native message fields for compatibility. - `isphere_receive_messages` now returns the contract-facing `ok`, `conversation`, `messages`, `next_cursor`, and `audit` envelope while retaining legacy parser-native message fields for compatibility.
- `isphere_receive_messages` accepts the read-contract argument set for the current local-readonly path: `conversation_id`, `query`, `since`, `limit`, `include_attachment_metadata`, `source_preference`, `preview`, and empty `cursor`. - `isphere_receive_messages` accepts the read-contract argument set for the current local-readonly path: `conversation_id`, `query`, `since`, `limit`, `include_attachment_metadata`, `source_preference`, `preview`, and empty `cursor`.
- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown because C23 found no validated mapping source yet. - `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown because C23/C24 found no validated DB read path yet.
- `isphere_receive_files` list mode accepts and validates the safe file-list contract args for the current local-readonly path while keeping download blocked. - `isphere_receive_files` list mode accepts and validates the safe file-list contract args for the current local-readonly path while keeping download blocked.
- N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures. - N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures.
- N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only. - N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only.
@@ -55,7 +55,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme
## Current loop ## Current loop
Current loop: `Stage C / Loop C24 - MsgLib.db readability/wrapper path precheck`. Current loop: `Stage C / Loop C25 - static .NET DB wrapper analysis`.
Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`. Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`.
@@ -91,10 +91,11 @@ Current loop output so far:
21. C21: `isphere_search_contacts` and `isphere_search_groups` now accept and validate safe search-contract args, while rejecting unsupported source/cursor cases. 21. C21: `isphere_search_contacts` and `isphere_search_groups` now accept and validate safe search-contract args, while rejecting unsupported source/cursor cases.
22. C22: `isphere_receive_files` list mode now accepts and validates safe contract args, rejects unsupported source/cursor/output/download cases, and keeps download blocked. 22. C22: `isphere_receive_files` list mode now accepts and validates safe contract args, rejects unsupported source/cursor/output/download cases, and keeps download blocked.
23. C23: display-name enrichment was blocked because current committed evidence and ignored metadata do not validate a JID-to-display-name/group-title source; `MsgLib.db` remains the most likely next source but inspected copies are not directly readable as SQLite. 23. C23: display-name enrichment was blocked because current committed evidence and ignored metadata do not validate a JID-to-display-name/group-title source; `MsgLib.db` remains the most likely next source but inspected copies are not directly readable as SQLite.
24. C24: `MsgLib.db` readability precheck found high-entropy non-SQLite files with a consistent wrapper/encryption signature; wrapper candidates include `HYHC.IMPP.DAL.dll`, `Utilities.Lib.SQLiteInteraction*.dll`, `Utilities.Lib.DBModel.dll`, and `RepairDatabase.exe`.
## Next business mainline ## Next business mainline
1. Run Stage C Loop C24: precheck why `MsgLib.db` is not directly readable as SQLite and identify the safest wrapper/decryption/read path before attempting display-name enrichment. 1. Run Stage C Loop C25: statically inspect the .NET DB wrapper assemblies to map table/model names and connection-string/password/decryption flow before any copied-DB read attempt.
2. Leave `isphere_send_message` blocked until bridge/API, UIA action-chain, or protocol connector evidence is validated. 2. Leave `isphere_send_message` blocked until bridge/API, UIA action-chain, or protocol connector evidence is validated.
3. Leave actual file download/cache mapping as a later connector node until a message-to-cache hash or client download source is validated. 3. Leave actual file download/cache mapping as a later connector node until a message-to-cache hash or client download source is validated.
4. Keep UIA selector/report work as fallback support, not as the primary roadmap. 4. Keep UIA selector/report work as fallback support, not as the primary roadmap.

View File

@@ -0,0 +1,85 @@
# MsgLib.db readability and wrapper-path precheck
Date: 2026-07-10
## Question
Why are the copied `MsgLib.db` files not directly readable as SQLite, and what is the safest next path for extracting contact/group display names and richer message metadata?
## Evidence checked
- Existing ignored metadata:
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/schema-inspection-summary.json`
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/archive-list.txt`
- New ignored C24 metadata:
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/c24-msglib-format-precheck.json`
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/c24-wrapper-string-precheck.json`
- Read-only extracted wrapper candidates under ignored:
- `runs/offline-evidence-intake/zyl-qqfile-20260709/extracted/c24-wrapper-candidates/`
No raw DB bytes, decrypted messages, or personal values are committed in this note.
## Findings
1. The three inspected `MsgLib.db` copies do not start with SQLite magic and contain no SQLite header within the first 1 MiB sample.
2. The inspected DB samples have high entropy and the same non-SQLite prefix shape across iSphere and IMPP account copies. This strongly suggests a consistent encrypted/wrapped DB format rather than a random corrupt copy.
3. The archive contains standard SQLite libraries, DB abstraction libraries, and a repair tool next to `MsgLib.db`:
- `System.Data.SQLite.dll`
- `Utilities.Lib.DBSQLite.dll`
- `Utilities.Lib.SQLiteInteraction.dll`
- `Utilities.Lib.SQLiteInteractionBase.dll`
- `Utilities.Lib.DBBase.dll`
- `Utilities.Lib.DBModel.dll`
- `HYHC.IMPP.DAL.dll`
- `RepairDatabase.exe`
4. String-only static scanning of wrapper candidates shows that `HYHC.IMPP.DAL.dll` and `RepairDatabase.exe` reference message/group DB behavior and candidate table/method names, including:
- `tblMsgGroupPersonMsg`
- `ImportOldDataBase`
- `GetGroupMessaeByID`
- `GetGroupRecentMsg`
- `UpdateGroupName`
- `SelectMsgGroupName`
- `SelectRecentGroupName`
- `SaveGroupInfomation`
- `groupJid`
- `set_MemberJid`
- `set_MsgGroupName`
- `set_MemberName`
- `get_naturalname`
5. `Utilities.Lib.SQLiteInteraction.dll` exposes `SQLiteConnectionStringBuilder` plus `get_Password` and `set_Password`.
6. `Utilities.Lib.DBModel.dll` exposes configuration concepts such as `IConStrDecryption`, `PasswordPropertyName`, and `DecryptionPropertyName`.
7. `smcrypto-*` and `BouncyCastle.Crypto.dll` are present, but C24 does not prove they are the actual `MsgLib.db` wrapper. They are supporting crypto candidates only.
## Decision
Do not treat `MsgLib.db` as ordinary SQLite.
The best current hypothesis is:
> `MsgLib.db` is an encrypted or wrapped SQLite database opened through the client DB abstraction layer, likely involving `Utilities.Lib.SQLiteInteraction*`, `Utilities.Lib.DBModel`, and `HYHC.IMPP.DAL`.
The next safe implementation path is not direct DB reads. It is static wrapper analysis first:
1. Inspect .NET metadata/IL for the DB wrapper assemblies.
2. Identify how connection strings/password/decryption are supplied.
3. Identify table names and model classes for message, contact, group, member, and display-name fields.
4. Only after that, attempt a read-only schema extraction on copied DB files under ignored paths.
## Next slice
C25 should perform a read-only static .NET wrapper analysis over the extracted candidate assemblies. It should produce table/model/source mappings only, not execute the client and not write to the original DB.
Suggested C25 focus:
- `HYHC.IMPP.DAL.dll`
- `Utilities.Lib.SQLiteInteraction.dll`
- `Utilities.Lib.DBModel.dll`
- `Utilities.Lib.DBSQLite.dll`
- `RepairDatabase.exe`
Expected output:
- candidate table names;
- candidate model/property names for contact/group/message metadata;
- likely connection-string/password/decryption flow;
- decision whether C26 can attempt a copied-DB read-only schema extraction.

View File

@@ -18,8 +18,8 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`
| MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice | | MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice |
| --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- |
| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; wrapped `MsgLib.db` after DB wrapper is solved | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence` | C20 supports and validates the current safe receive-message contract args; default source remains empty when unset | apply safe contract args to contact/group search | | `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; wrapped `MsgLib.db` after DB wrapper is solved | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence` | C20 supports and validates the current safe receive-message contract args; default source remains empty when unset | apply safe contract args to contact/group search |
| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | decrypted roster/contact stanzas from `Smark.SendReceive`; `MsgLib.db` contact/conversation tables after DB wrapper is solved; UIA helper source if display names are missing | `docs/source-discovery/2026-07-10-display-name-source-precheck.md` | C23 blocked display-name enrichment: no validated JID-to-display-name source yet; JID-only display/account fallback remains honest | precheck `MsgLib.db` readability/wrapper path | | `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | decrypted roster/contact stanzas from `Smark.SendReceive`; `MsgLib.db` contact/conversation tables after DB wrapper is solved; UIA helper source if display names are missing | `docs/source-discovery/2026-07-10-msglib-readability-precheck.md` | C24 confirms `MsgLib.db` is not ordinary SQLite and likely needs client DB wrapper/static .NET analysis before display-name extraction | static wrapper analysis for table/model/password flow |
| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | `MsgLib.db` group/conversation tables after DB wrapper is solved; UIA helper source if group display names are missing | `docs/source-discovery/2026-07-10-display-name-source-precheck.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C23 blocked display-name/member enrichment: no validated group-title/member source yet; JID-only display fallback remains honest | precheck `MsgLib.db` readability/wrapper path | | `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | `MsgLib.db` group/conversation tables after DB wrapper is solved; UIA helper source if group display names are missing | `docs/source-discovery/2026-07-10-msglib-readability-precheck.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C24 finds group table/model candidates in wrapper strings but no validated copied-DB read path yet; JID-only display fallback remains honest | static wrapper analysis for table/model/password flow |
| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence | | `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence |
| `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration | | `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration |
| `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated | | `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated |

View File

@@ -1925,15 +1925,15 @@ Evidence precheck result:
- If not identifiable, record the blocker and shift to another business-read hardening node. - If not identifiable, record the blocker and shift to another business-read hardening node.
- Do not add write/send/file-download behavior. - Do not add write/send/file-download behavior.
- [ ] **Step 1: Inspect safe DB metadata** - [x] **Step 1: Inspect safe DB metadata**
Use file headers, existing summaries, and committed notes only. Avoid committing raw bytes. Use file headers, existing summaries, and committed notes only. Avoid committing raw bytes.
- [ ] **Step 2: Record likely wrapper/read path** - [x] **Step 2: Record likely wrapper/read path**
Write a source-discovery note with the decision and next implementation slice. Write a source-discovery note with the decision and next implementation slice.
- [ ] **Step 3: Verify and commit docs** - [x] **Step 3: Verify and commit docs**
Run: Run:
@@ -1947,6 +1947,72 @@ Expected: all exit 0. Commit exact C24 paths only and rewrite the next loop from
--- ---
## Loop C24 Result
Evidence precheck result:
- Three inspected `MsgLib.db` copies do not start with SQLite magic and contain no SQLite header in the first 1 MiB sample.
- The files have high entropy and share the same non-SQLite prefix shape across iSphere and IMPP copies, so the likely issue is an encrypted/wrapped DB format rather than a random corrupt copy.
- Archive metadata shows SQLite and DB wrapper assemblies next to the client:
- `System.Data.SQLite.dll`
- `Utilities.Lib.DBSQLite.dll`
- `Utilities.Lib.SQLiteInteraction.dll`
- `Utilities.Lib.SQLiteInteractionBase.dll`
- `Utilities.Lib.DBBase.dll`
- `Utilities.Lib.DBModel.dll`
- `HYHC.IMPP.DAL.dll`
- `RepairDatabase.exe`
- Ignored string-only scans of extracted wrapper candidates show relevant method/table/model hints including `tblMsgGroupPersonMsg`, `ImportOldDataBase`, `GetGroupRecentMsg`, `UpdateGroupName`, `SelectMsgGroupName`, `SelectRecentGroupName`, `groupJid`, `set_MemberJid`, `set_MsgGroupName`, `set_MemberName`, and `get_naturalname`.
- `Utilities.Lib.SQLiteInteraction.dll` exposes SQLite connection builder plus password accessors; `Utilities.Lib.DBModel.dll` exposes connection-string decryption/password configuration concepts.
- Decision: do not read `MsgLib.db` as ordinary SQLite. The next safe route is static .NET wrapper analysis before any copied-DB read attempt.
---
## Loop C25: Static .NET DB wrapper analysis
**Goal:** Statically inspect the extracted DB wrapper assemblies to map candidate table/model names and connection-string/password/decryption flow for future read-only `MsgLib.db` access.
**Planned files:**
- Modify: `docs/source-discovery/capability-source-matrix.md`
- Modify: `docs/current-status-card.md`
- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`
- Optionally create a focused source-discovery note under `docs/source-discovery/`.
- Optionally create ignored metadata under `runs/`; do not commit binaries, raw DB bytes, decrypted content, or personal values.
**Planned behavior:**
- Inspect .NET assembly metadata/IL read-only for:
- `HYHC.IMPP.DAL.dll`
- `Utilities.Lib.SQLiteInteraction.dll`
- `Utilities.Lib.DBModel.dll`
- `Utilities.Lib.DBSQLite.dll`
- `RepairDatabase.exe`
- Extract only table/model/method/property names and connection/decryption flow summaries.
- Decide whether C26 can attempt copied-DB read-only schema extraction or must stay blocked.
- Do not execute client binaries or modify DB files.
- Do not add write/send/file-download behavior.
- [ ] **Step 1: Generate static wrapper metadata**
Use read-only metadata/IL tools or string extraction. Keep output under ignored metadata.
- [ ] **Step 2: Write source-discovery conclusion**
Record likely tables/models and the safest next read path.
- [ ] **Step 3: Verify and commit docs**
Run:
```powershell
git diff --check
go test ./...
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/verify-go-mcp.ps1
```
Expected: all exit 0. Commit exact C25 paths only and rewrite the next loop from actual results.
---
## Self-Review ## Self-Review
- Spec coverage: the plan follows the user's cyclic requirement: plan first, implement one loop at a time, update the next loop after each result, and ask only on blockers. - Spec coverage: the plan follows the user's cyclic requirement: plan first, implement one loop at a time, update the next loop after each result, and ask only on blockers.