docs: identify msglib wrapper analysis path
This commit is contained in:
zhaoyilun
@@ -27,7 +27,7 @@ Remote base before local roadmap commits: `b2d839e Merge branch 'codex/n15-repor
|
||||
- `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, eight-tool MCP surface, default empty-source receive/contact/group/file calls, synthetic configured-file receive/contact/group/file smoke, and synthetic configured-directory receive/contact/group/file smoke.
|
||||
- `isphere_receive_messages` now returns the contract-facing `ok`, `conversation`, `messages`, `next_cursor`, and `audit` envelope while retaining legacy parser-native message fields for compatibility.
|
||||
- `isphere_receive_messages` accepts the read-contract argument set for the current local-readonly path: `conversation_id`, `query`, `since`, `limit`, `include_attachment_metadata`, `source_preference`, `preview`, and empty `cursor`.
|
||||
- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown because C23 found no validated mapping source yet.
|
||||
- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown because C23/C24 found no validated DB read path yet.
|
||||
- `isphere_receive_files` list mode accepts and validates the safe file-list contract args for the current local-readonly path while keeping download blocked.
|
||||
- N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures.
|
||||
- N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only.
|
||||
@@ -55,7 +55,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme
|
||||
|
||||
## Current loop
|
||||
|
||||
Current loop: `Stage C / Loop C24 - MsgLib.db readability/wrapper path precheck`.
|
||||
Current loop: `Stage C / Loop C25 - static .NET DB wrapper analysis`.
|
||||
|
||||
Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`.
|
||||
|
||||
@@ -91,10 +91,11 @@ Current loop output so far:
|
||||
21. C21: `isphere_search_contacts` and `isphere_search_groups` now accept and validate safe search-contract args, while rejecting unsupported source/cursor cases.
|
||||
22. C22: `isphere_receive_files` list mode now accepts and validates safe contract args, rejects unsupported source/cursor/output/download cases, and keeps download blocked.
|
||||
23. C23: display-name enrichment was blocked because current committed evidence and ignored metadata do not validate a JID-to-display-name/group-title source; `MsgLib.db` remains the most likely next source but inspected copies are not directly readable as SQLite.
|
||||
24. C24: `MsgLib.db` readability precheck found high-entropy non-SQLite files with a consistent wrapper/encryption signature; wrapper candidates include `HYHC.IMPP.DAL.dll`, `Utilities.Lib.SQLiteInteraction*.dll`, `Utilities.Lib.DBModel.dll`, and `RepairDatabase.exe`.
|
||||
|
||||
## Next business mainline
|
||||
|
||||
1. Run Stage C Loop C24: precheck why `MsgLib.db` is not directly readable as SQLite and identify the safest wrapper/decryption/read path before attempting display-name enrichment.
|
||||
1. Run Stage C Loop C25: statically inspect the .NET DB wrapper assemblies to map table/model names and connection-string/password/decryption flow before any copied-DB read attempt.
|
||||
2. Leave `isphere_send_message` blocked until bridge/API, UIA action-chain, or protocol connector evidence is validated.
|
||||
3. Leave actual file download/cache mapping as a later connector node until a message-to-cache hash or client download source is validated.
|
||||
4. Keep UIA selector/report work as fallback support, not as the primary roadmap.
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
# MsgLib.db readability and wrapper-path precheck
|
||||
|
||||
Date: 2026-07-10
|
||||
|
||||
## Question
|
||||
|
||||
Why are the copied `MsgLib.db` files not directly readable as SQLite, and what is the safest next path for extracting contact/group display names and richer message metadata?
|
||||
|
||||
## Evidence checked
|
||||
|
||||
- Existing ignored metadata:
|
||||
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/schema-inspection-summary.json`
|
||||
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/archive-list.txt`
|
||||
- New ignored C24 metadata:
|
||||
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/c24-msglib-format-precheck.json`
|
||||
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/c24-wrapper-string-precheck.json`
|
||||
- Read-only extracted wrapper candidates under ignored:
|
||||
- `runs/offline-evidence-intake/zyl-qqfile-20260709/extracted/c24-wrapper-candidates/`
|
||||
|
||||
No raw DB bytes, decrypted messages, or personal values are committed in this note.
|
||||
|
||||
## Findings
|
||||
|
||||
1. The three inspected `MsgLib.db` copies do not start with SQLite magic and contain no SQLite header within the first 1 MiB sample.
|
||||
2. The inspected DB samples have high entropy and the same non-SQLite prefix shape across iSphere and IMPP account copies. This strongly suggests a consistent encrypted/wrapped DB format rather than a random corrupt copy.
|
||||
3. The archive contains standard SQLite libraries, DB abstraction libraries, and a repair tool next to `MsgLib.db`:
|
||||
- `System.Data.SQLite.dll`
|
||||
- `Utilities.Lib.DBSQLite.dll`
|
||||
- `Utilities.Lib.SQLiteInteraction.dll`
|
||||
- `Utilities.Lib.SQLiteInteractionBase.dll`
|
||||
- `Utilities.Lib.DBBase.dll`
|
||||
- `Utilities.Lib.DBModel.dll`
|
||||
- `HYHC.IMPP.DAL.dll`
|
||||
- `RepairDatabase.exe`
|
||||
4. String-only static scanning of wrapper candidates shows that `HYHC.IMPP.DAL.dll` and `RepairDatabase.exe` reference message/group DB behavior and candidate table/method names, including:
|
||||
- `tblMsgGroupPersonMsg`
|
||||
- `ImportOldDataBase`
|
||||
- `GetGroupMessaeByID`
|
||||
- `GetGroupRecentMsg`
|
||||
- `UpdateGroupName`
|
||||
- `SelectMsgGroupName`
|
||||
- `SelectRecentGroupName`
|
||||
- `SaveGroupInfomation`
|
||||
- `groupJid`
|
||||
- `set_MemberJid`
|
||||
- `set_MsgGroupName`
|
||||
- `set_MemberName`
|
||||
- `get_naturalname`
|
||||
5. `Utilities.Lib.SQLiteInteraction.dll` exposes `SQLiteConnectionStringBuilder` plus `get_Password` and `set_Password`.
|
||||
6. `Utilities.Lib.DBModel.dll` exposes configuration concepts such as `IConStrDecryption`, `PasswordPropertyName`, and `DecryptionPropertyName`.
|
||||
7. `smcrypto-*` and `BouncyCastle.Crypto.dll` are present, but C24 does not prove they are the actual `MsgLib.db` wrapper. They are supporting crypto candidates only.
|
||||
|
||||
## Decision
|
||||
|
||||
Do not treat `MsgLib.db` as ordinary SQLite.
|
||||
|
||||
The best current hypothesis is:
|
||||
|
||||
> `MsgLib.db` is an encrypted or wrapped SQLite database opened through the client DB abstraction layer, likely involving `Utilities.Lib.SQLiteInteraction*`, `Utilities.Lib.DBModel`, and `HYHC.IMPP.DAL`.
|
||||
|
||||
The next safe implementation path is not direct DB reads. It is static wrapper analysis first:
|
||||
|
||||
1. Inspect .NET metadata/IL for the DB wrapper assemblies.
|
||||
2. Identify how connection strings/password/decryption are supplied.
|
||||
3. Identify table names and model classes for message, contact, group, member, and display-name fields.
|
||||
4. Only after that, attempt a read-only schema extraction on copied DB files under ignored paths.
|
||||
|
||||
## Next slice
|
||||
|
||||
C25 should perform a read-only static .NET wrapper analysis over the extracted candidate assemblies. It should produce table/model/source mappings only, not execute the client and not write to the original DB.
|
||||
|
||||
Suggested C25 focus:
|
||||
|
||||
- `HYHC.IMPP.DAL.dll`
|
||||
- `Utilities.Lib.SQLiteInteraction.dll`
|
||||
- `Utilities.Lib.DBModel.dll`
|
||||
- `Utilities.Lib.DBSQLite.dll`
|
||||
- `RepairDatabase.exe`
|
||||
|
||||
Expected output:
|
||||
|
||||
- candidate table names;
|
||||
- candidate model/property names for contact/group/message metadata;
|
||||
- likely connection-string/password/decryption flow;
|
||||
- decision whether C26 can attempt a copied-DB read-only schema extraction.
|
||||
@@ -18,8 +18,8 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`
|
||||
| MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; wrapped `MsgLib.db` after DB wrapper is solved | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence` | C20 supports and validates the current safe receive-message contract args; default source remains empty when unset | apply safe contract args to contact/group search |
|
||||
| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | decrypted roster/contact stanzas from `Smark.SendReceive`; `MsgLib.db` contact/conversation tables after DB wrapper is solved; UIA helper source if display names are missing | `docs/source-discovery/2026-07-10-display-name-source-precheck.md` | C23 blocked display-name enrichment: no validated JID-to-display-name source yet; JID-only display/account fallback remains honest | precheck `MsgLib.db` readability/wrapper path |
|
||||
| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | `MsgLib.db` group/conversation tables after DB wrapper is solved; UIA helper source if group display names are missing | `docs/source-discovery/2026-07-10-display-name-source-precheck.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C23 blocked display-name/member enrichment: no validated group-title/member source yet; JID-only display fallback remains honest | precheck `MsgLib.db` readability/wrapper path |
|
||||
| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | decrypted roster/contact stanzas from `Smark.SendReceive`; `MsgLib.db` contact/conversation tables after DB wrapper is solved; UIA helper source if display names are missing | `docs/source-discovery/2026-07-10-msglib-readability-precheck.md` | C24 confirms `MsgLib.db` is not ordinary SQLite and likely needs client DB wrapper/static .NET analysis before display-name extraction | static wrapper analysis for table/model/password flow |
|
||||
| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | `MsgLib.db` group/conversation tables after DB wrapper is solved; UIA helper source if group display names are missing | `docs/source-discovery/2026-07-10-msglib-readability-precheck.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C24 finds group table/model candidates in wrapper strings but no validated copied-DB read path yet; JID-only display fallback remains honest | static wrapper analysis for table/model/password flow |
|
||||
| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence |
|
||||
| `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration |
|
||||
| `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated |
|
||||
|
||||
@@ -1925,15 +1925,15 @@ Evidence precheck result:
|
||||
- If not identifiable, record the blocker and shift to another business-read hardening node.
|
||||
- Do not add write/send/file-download behavior.
|
||||
|
||||
- [ ] **Step 1: Inspect safe DB metadata**
|
||||
- [x] **Step 1: Inspect safe DB metadata**
|
||||
|
||||
Use file headers, existing summaries, and committed notes only. Avoid committing raw bytes.
|
||||
|
||||
- [ ] **Step 2: Record likely wrapper/read path**
|
||||
- [x] **Step 2: Record likely wrapper/read path**
|
||||
|
||||
Write a source-discovery note with the decision and next implementation slice.
|
||||
|
||||
- [ ] **Step 3: Verify and commit docs**
|
||||
- [x] **Step 3: Verify and commit docs**
|
||||
|
||||
Run:
|
||||
|
||||
@@ -1947,6 +1947,72 @@ Expected: all exit 0. Commit exact C24 paths only and rewrite the next loop from
|
||||
|
||||
---
|
||||
|
||||
## Loop C24 Result
|
||||
|
||||
Evidence precheck result:
|
||||
|
||||
- Three inspected `MsgLib.db` copies do not start with SQLite magic and contain no SQLite header in the first 1 MiB sample.
|
||||
- The files have high entropy and share the same non-SQLite prefix shape across iSphere and IMPP copies, so the likely issue is an encrypted/wrapped DB format rather than a random corrupt copy.
|
||||
- Archive metadata shows SQLite and DB wrapper assemblies next to the client:
|
||||
- `System.Data.SQLite.dll`
|
||||
- `Utilities.Lib.DBSQLite.dll`
|
||||
- `Utilities.Lib.SQLiteInteraction.dll`
|
||||
- `Utilities.Lib.SQLiteInteractionBase.dll`
|
||||
- `Utilities.Lib.DBBase.dll`
|
||||
- `Utilities.Lib.DBModel.dll`
|
||||
- `HYHC.IMPP.DAL.dll`
|
||||
- `RepairDatabase.exe`
|
||||
- Ignored string-only scans of extracted wrapper candidates show relevant method/table/model hints including `tblMsgGroupPersonMsg`, `ImportOldDataBase`, `GetGroupRecentMsg`, `UpdateGroupName`, `SelectMsgGroupName`, `SelectRecentGroupName`, `groupJid`, `set_MemberJid`, `set_MsgGroupName`, `set_MemberName`, and `get_naturalname`.
|
||||
- `Utilities.Lib.SQLiteInteraction.dll` exposes SQLite connection builder plus password accessors; `Utilities.Lib.DBModel.dll` exposes connection-string decryption/password configuration concepts.
|
||||
- Decision: do not read `MsgLib.db` as ordinary SQLite. The next safe route is static .NET wrapper analysis before any copied-DB read attempt.
|
||||
|
||||
---
|
||||
|
||||
## Loop C25: Static .NET DB wrapper analysis
|
||||
|
||||
**Goal:** Statically inspect the extracted DB wrapper assemblies to map candidate table/model names and connection-string/password/decryption flow for future read-only `MsgLib.db` access.
|
||||
|
||||
**Planned files:**
|
||||
- Modify: `docs/source-discovery/capability-source-matrix.md`
|
||||
- Modify: `docs/current-status-card.md`
|
||||
- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`
|
||||
- Optionally create a focused source-discovery note under `docs/source-discovery/`.
|
||||
- Optionally create ignored metadata under `runs/`; do not commit binaries, raw DB bytes, decrypted content, or personal values.
|
||||
|
||||
**Planned behavior:**
|
||||
- Inspect .NET assembly metadata/IL read-only for:
|
||||
- `HYHC.IMPP.DAL.dll`
|
||||
- `Utilities.Lib.SQLiteInteraction.dll`
|
||||
- `Utilities.Lib.DBModel.dll`
|
||||
- `Utilities.Lib.DBSQLite.dll`
|
||||
- `RepairDatabase.exe`
|
||||
- Extract only table/model/method/property names and connection/decryption flow summaries.
|
||||
- Decide whether C26 can attempt copied-DB read-only schema extraction or must stay blocked.
|
||||
- Do not execute client binaries or modify DB files.
|
||||
- Do not add write/send/file-download behavior.
|
||||
|
||||
- [ ] **Step 1: Generate static wrapper metadata**
|
||||
|
||||
Use read-only metadata/IL tools or string extraction. Keep output under ignored metadata.
|
||||
|
||||
- [ ] **Step 2: Write source-discovery conclusion**
|
||||
|
||||
Record likely tables/models and the safest next read path.
|
||||
|
||||
- [ ] **Step 3: Verify and commit docs**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git diff --check
|
||||
go test ./...
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/verify-go-mcp.ps1
|
||||
```
|
||||
|
||||
Expected: all exit 0. Commit exact C25 paths only and rewrite the next loop from actual results.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: the plan follows the user's cyclic requirement: plan first, implement one loop at a time, update the next loop after each result, and ask only on blockers.
|
||||
|
||||
Reference in New Issue
Block a user