docs: validate msglib readonly schema path
This commit is contained in:
zhaoyilun
@@ -0,0 +1,115 @@
|
||||
# MsgLib.db copied read-only schema extraction
|
||||
|
||||
Date: 2026-07-10
|
||||
|
||||
## Question
|
||||
|
||||
Can copied `MsgLib.db` files be opened read-only with the wrapper password recovered in C25, and which schema fields should drive the next business read implementation?
|
||||
|
||||
## Evidence checked
|
||||
|
||||
Ignored C26 artifacts:
|
||||
|
||||
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/c26-schema-probe-x86/`
|
||||
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/c26-msglib-schema-probe-x86.json`
|
||||
|
||||
Probe method:
|
||||
|
||||
- referenced the bundled `System.Data.SQLite.dll` from the extracted iSphere client files;
|
||||
- opened copied `MsgLib.db` files only;
|
||||
- used the C25 recovered password `123`;
|
||||
- forced read-only mode in the SQLite connection string;
|
||||
- queried only `sqlite_master`, `PRAGMA table_info(...)`, and safe target-table row counts;
|
||||
- did not dump message bodies, contact values, file paths, or raw rows.
|
||||
|
||||
## Runtime finding
|
||||
|
||||
- A 64-bit probe failed at CLR/SQLite mixed-mode loading.
|
||||
- A 32-bit .NET Framework probe succeeded.
|
||||
- Successful provider: `System.Data.SQLite`, version observed through the opened DB runtime as SQLite `3.6.23.1`.
|
||||
|
||||
This means the production path should not assume Go can open this DB directly. The practical route is a dedicated x86 .NET read-only sidecar/helper that Go MCP calls through a bounded JSON contract.
|
||||
|
||||
## Open result
|
||||
|
||||
All three copied `MsgLib.db` candidates opened successfully with `Password=123` in read-only mode.
|
||||
|
||||
| Copied DB | Open result | Table count | Notable safe counts |
|
||||
| --- | --- | ---: | --- |
|
||||
| DB A, about 12.2 MB | ok | 13 | `tblMsgGroupPersonMsg=2884`, `tblPersonMsg=2132`, `tblRecent=52`, `TD_SystemMessageRecord=469` |
|
||||
| DB B, about 2.84 MB | ok | 23 | `tblMsgGroupPersonMsg=1048`, `tblPersonMsg=1209`, `tblRecent=80`, `tblChatLevel=105`, `TD_WorkGroupAuth=2`, `TD_SystemMessageRecord=314` |
|
||||
| DB C, about 2.78 MB | ok | 13 | `tblMsgGroupPersonMsg=796`, `tblPersonMsg=1932`, `tblRecent=51`, `TD_SystemMessageRecord=228` |
|
||||
|
||||
The counts above are schema-validation counts only. No row values were extracted into committed docs.
|
||||
|
||||
## Confirmed business schema map
|
||||
|
||||
### Contacts and display names
|
||||
|
||||
Validated tables/columns:
|
||||
|
||||
- `TD_Roster`: `JId`, `Domain`, `Nick`, `Ask`, `Subscription`, `Show`, `Status`, `GroupName`.
|
||||
- `TD_CustomEffigy`: `PersonJid`, `PersonName`, `PersonSex`, `HeadImageSize`, `LocalPath`, `ServerUpdateTime`.
|
||||
- `tblRecent`: `Id`, `PersonId`, `PersonName`, `ActionTime`, `FType`.
|
||||
- `tblChatLevel`: present in the richer DB and already known from C25 model fields as `PersonId`, `PersonName`.
|
||||
- `tblPersonMsg`: `SndPerson`, `SndPersonId`, `RecvPerson`, `RecvPersonId`, `ObjPerson`, `ObjPersonId`, plus message metadata fields.
|
||||
|
||||
Recommended contact display-name priority for a future read source:
|
||||
|
||||
1. `TD_Roster.Nick` by `TD_Roster.JId` when present.
|
||||
2. `TD_CustomEffigy.PersonName` by `PersonJid` when present.
|
||||
3. `tblRecent.PersonName` by `PersonId` for recent conversations.
|
||||
4. `tblPersonMsg` sender/receiver/object display fields as fallback.
|
||||
5. JID-derived fallback when DB display fields are absent.
|
||||
|
||||
### Groups and member names
|
||||
|
||||
Validated tables/columns:
|
||||
|
||||
- `tblMsgGroupPersonMsg`: `MsgGroupId`, `MsgGroupName`, `SessionTitle`, `SessionPersonId`, `SessionPersonName`, `GroupType`, plus message metadata fields.
|
||||
- `TD_WorkGroupAuth`: `SessionID`, `SessionStatus`, `SessionType`, `ApplyInfo`, `GroupJID`, `GroupName`, `SendPersonSex`, `ApplyDateTime`; richer DB also has `ChildGroupJid`.
|
||||
- `tblRecent`: stores group recent entries through triggers using `PersonId = MsgGroupId`, `PersonName = MsgGroupName`, and `FType = GroupType`.
|
||||
|
||||
Recommended group display/member priority:
|
||||
|
||||
1. `tblMsgGroupPersonMsg.MsgGroupName` by `MsgGroupId`.
|
||||
2. `tblRecent.PersonName` by `PersonId` where `FType` indicates group/discussion.
|
||||
3. `TD_WorkGroupAuth.GroupName` by `GroupJID` for work-group authorization records.
|
||||
4. `tblMsgGroupPersonMsg.SessionPersonId/SessionPersonName` for member-name enrichment.
|
||||
5. JID-derived fallback when DB display fields are absent.
|
||||
|
||||
### Receive messages
|
||||
|
||||
Validated tables/columns:
|
||||
|
||||
- `tblPersonMsg`: one-to-one message table, including ids, sender/receiver/object identities, text/body/content/version/read fields, timestamps, and receipt state.
|
||||
- `tblMsgGroupPersonMsg`: group/discussion message table, including ids, sender identities, group ids/names, session title/member fields, message body/content/version/read fields, and timestamps.
|
||||
- `TD_SystemMessageRecord`: system/auth message metadata.
|
||||
|
||||
C27 should not read message bodies yet. The next code loop should first add a schema/display-name sidecar and only later add DB-backed message listing with strict limit/redaction controls.
|
||||
|
||||
### File metadata
|
||||
|
||||
Validated tables/columns:
|
||||
|
||||
- `TD_ReceiveFileRecord`: `ReceiveMsgGuid`, `FileName`, `FileSize`, `FilePath`, `SourceID`, `SourceName`, `FileType`, `SendPersonJid`, `SendPersonName`, `SendTime`.
|
||||
- `TD_SendFileRecord`: `SendMsgGuid`, `FileName`, `FileSize`, `FilePath`, `SourceID`, `SourceName`, `FileType`, `ReceivePersonJid`, `ReceivePersonName`, `ReceiveTime`.
|
||||
|
||||
File download remains blocked until cache/download mapping is separately validated.
|
||||
|
||||
## Decision
|
||||
|
||||
C26 validates the DB path:
|
||||
|
||||
> `MsgLib.db` can be opened from copied evidence with the bundled 32-bit `System.Data.SQLite.dll`, `Password=123`, and read-only mode.
|
||||
|
||||
The project can now move from log-only read support toward DB-backed contact/group display-name enrichment. Because the working provider path is 32-bit .NET, the next implementation should be a bounded x86 .NET sidecar/helper rather than direct Go SQLite access.
|
||||
|
||||
## Next slice
|
||||
|
||||
Loop C27 should implement the first production-shaped DB bridge as a narrow helper/sidecar contract, not as broad message extraction:
|
||||
|
||||
1. Build a committed contract for a read-only `MsgLib.db` schema/display-name sidecar.
|
||||
2. Start with safe operations such as `msglib_self_check` and `msglib_list_display_sources` or equivalent.
|
||||
3. Return only table availability, column availability, and display-name candidate maps with strict limits.
|
||||
4. Keep message-body reads, file downloads, sends, writes, and DB mutation out of scope.
|
||||
@@ -17,9 +17,9 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`
|
||||
|
||||
| MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; copied `MsgLib.db` after C26 read-only schema/open validation | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C20 supports and validates the current safe receive-message contract args; C25 recovered the likely `MsgLib.db` wrapper password/connection flow but DB open is not validated yet | C26 copied-DB read-only schema extraction |
|
||||
| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | `MsgLib.db` tables after C26 read-only validation: `TD_Roster`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C25 recovered the static wrapper path and candidate contact/display-name tables; direct copied-DB read is still unvalidated | C26 copied-DB read-only schema extraction with recovered wrapper password |
|
||||
| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | `MsgLib.db` tables after C26 read-only validation: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C25 recovered the static wrapper path and candidate group/member display fields; direct copied-DB read is still unvalidated | C26 copied-DB read-only schema extraction with recovered wrapper password |
|
||||
| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; copied `MsgLib.db` through a future x86 read-only sidecar/helper | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C20 supports and validates the current safe receive-message contract args; C26 validates copied `MsgLib.db` read-only open/schema, but DB-backed message listing is not implemented yet | C27 bounded x86 MsgLib sidecar/helper contract for safe display/schema reads |
|
||||
| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md` | C26 validates read-only DB open/schema with the recovered wrapper password; product code still needs a bounded x86 sidecar/helper before DB-backed display names can be exposed | C27 bounded x86 MsgLib sidecar/helper contract |
|
||||
| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C26 validates read-only DB open/schema with the recovered wrapper password; product code still needs a bounded x86 sidecar/helper before DB-backed group/member names can be exposed | C27 bounded x86 MsgLib sidecar/helper contract |
|
||||
| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence |
|
||||
| `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration |
|
||||
| `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated |
|
||||
@@ -39,4 +39,4 @@ Start Stage C with a narrow log-backed `isphere_receive_messages` source abstrac
|
||||
|
||||
## Deferred source work
|
||||
|
||||
`MsgLib.db` is no longer treated as ordinary SQLite. C25 recovered the likely wrapper route (`System.Data.SQLite` plus DB config password `123`), but C26 must still validate a copied read-only DB open and schema extraction before any DB-backed production read source is implemented.
|
||||
`MsgLib.db` has a validated copied read-only open path through the bundled 32-bit `System.Data.SQLite.dll` and password `123`. Because the working provider path is x86 .NET, DB-backed production reads should enter through a bounded sidecar/helper contract rather than direct Go SQLite access.
|
||||
|
||||
Reference in New Issue
Block a user