admin/isphere-ai-bridge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add msglib readonly sidecar

Browse Source
This commit is contained in:
zhaoyilun
2026-07-10 02:45:55 +08:00
parent b442138e5c
commit 5b05228ba8
8 changed files with 849 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/current-status-card.md
View File

@@ -27,8 +27,9 @@ Remote base before local roadmap commits: `b2d839e Merge branch 'codex/n15-repor
- `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, eight-tool MCP surface, default empty-source receive/contact/group/file calls, synthetic configured-file receive/contact/group/file smoke, and synthetic configured-directory receive/contact/group/file smoke. - `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, eight-tool MCP surface, default empty-source receive/contact/group/file calls, synthetic configured-file receive/contact/group/file smoke, and synthetic configured-directory receive/contact/group/file smoke.
- `isphere_receive_messages` now returns the contract-facing `ok`, `conversation`, `messages`, `next_cursor`, and `audit` envelope while retaining legacy parser-native message fields for compatibility. - `isphere_receive_messages` now returns the contract-facing `ok`, `conversation`, `messages`, `next_cursor`, and `audit` envelope while retaining legacy parser-native message fields for compatibility.
- `isphere_receive_messages` accepts the read-contract argument set for the current local-readonly path: `conversation_id`, `query`, `since`, `limit`, `include_attachment_metadata`, `source_preference`, `preview`, and empty `cursor`. - `isphere_receive_messages` accepts the read-contract argument set for the current local-readonly path: `conversation_id`, `query`, `since`, `limit`, `include_attachment_metadata`, `source_preference`, `preview`, and empty `cursor`.
- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown until a bounded x86 `MsgLib.db` sidecar/helper is implemented. - `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown until the new x86 `MsgLibReadSidecar` is wrapped from Go and integrated.
- `isphere_receive_files` list mode accepts and validates the safe file-list contract args for the current local-readonly path while keeping download blocked. - `isphere_receive_files` list mode accepts and validates the safe file-list contract args for the current local-readonly path while keeping download blocked.
- `native/MsgLibReadSidecar` now provides a bounded x86 .NET read-only boundary for `MsgLib.db` schema/display-source checks; it is not yet wired into Go MCP tools.
- N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures. - N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures.
- N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only. - N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only.
@@ -55,7 +56,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme
## Current loop ## Current loop
Current loop: `Stage C / Loop C27 - bounded MsgLib sidecar/helper contract`. Current loop: `Stage C / Loop C28 - Go MsgLib sidecar client wrapper`.
Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`. Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`.
@@ -94,10 +95,11 @@ Current loop output so far:
24. C24: `MsgLib.db` readability precheck found high-entropy non-SQLite files with a consistent wrapper/encryption signature; wrapper candidates include `HYHC.IMPP.DAL.dll`, `Utilities.Lib.SQLiteInteraction*.dll`, `Utilities.Lib.DBModel.dll`, and `RepairDatabase.exe`. 24. C24: `MsgLib.db` readability precheck found high-entropy non-SQLite files with a consistent wrapper/encryption signature; wrapper candidates include `HYHC.IMPP.DAL.dll`, `Utilities.Lib.SQLiteInteraction*.dll`, `Utilities.Lib.DBModel.dll`, and `RepairDatabase.exe`.
25. C25: static .NET wrapper analysis recovered the DAL open path: `HYHC.IMPP.DAL.IMPPDAL` configures `System.Data.SQLite` through `Utilities.Lib.DBSQLite`, sets `DatabaseName` to the DB path and `Password` to `123`, and exposes candidate display/message tables such as `TD_Roster`, `tblRecent`, `tblPersonMsg`, `tblMsgGroupPersonMsg`, and `TD_WorkGroupAuth`. 25. C25: static .NET wrapper analysis recovered the DAL open path: `HYHC.IMPP.DAL.IMPPDAL` configures `System.Data.SQLite` through `Utilities.Lib.DBSQLite`, sets `DatabaseName` to the DB path and `Password` to `123`, and exposes candidate display/message tables such as `TD_Roster`, `tblRecent`, `tblPersonMsg`, `tblMsgGroupPersonMsg`, and `TD_WorkGroupAuth`.
26. C26: copied `MsgLib.db` schema extraction succeeded through a 32-bit .NET `System.Data.SQLite` read-only probe with password `123`; all three copied DB candidates opened, with confirmed contact/group/message/file schema tables. 26. C26: copied `MsgLib.db` schema extraction succeeded through a 32-bit .NET `System.Data.SQLite` read-only probe with password `123`; all three copied DB candidates opened, with confirmed contact/group/message/file schema tables.
27. C27: added `native/MsgLibReadSidecar` with protocol `isphere.msglib.v1`, x86 build/verify scripts, and safe `self_check`, `schema_summary`, and `display_sources` operations; evidence-backed smoke returned 8 display-source candidates without row/message values.
## Next business mainline ## Next business mainline
1. Run Stage C Loop C27: define and implement the first bounded x86 `MsgLib.db` sidecar/helper contract for schema/display-name reads; keep message bodies, file downloads, sends, writes, and DB mutation out of scope. 1. Run Stage C Loop C28: add a Go client wrapper for `MsgLibReadSidecar` using env-configured sidecar/SQLite/DB paths, with fake-sidecar tests first; do not expose a new MCP tool or read message bodies yet.
2. Leave `isphere_send_message` blocked until bridge/API, UIA action-chain, or protocol connector evidence is validated. 2. Leave `isphere_send_message` blocked until bridge/API, UIA action-chain, or protocol connector evidence is validated.
3. Leave actual file download/cache mapping as a later connector node until a message-to-cache hash or client download source is validated. 3. Leave actual file download/cache mapping as a later connector node until a message-to-cache hash or client download source is validated.
4. Keep UIA selector/report work as fallback support, not as the primary roadmap. 4. Keep UIA selector/report work as fallback support, not as the primary roadmap.

146
docs/msglib-read-sidecar-contract.md Normal file
View File

@@ -0,0 +1,146 @@
# MsgLib read-only sidecar contract
Date: 2026-07-10
## Purpose
`MsgLibReadSidecar` is the bounded 32-bit .NET process used by the Go MCP layer to access copied or operator-local `MsgLib.db` files through the client-compatible `System.Data.SQLite` provider.
It exists because C26 proved the database opens with the bundled 32-bit SQLite provider and password `123`, while a 64-bit probe fails at provider loading. Go should coordinate the process; the sidecar owns only the narrow Windows/.NET DB read boundary.
## Protocol
- Protocol id: `isphere.msglib.v1`
- Transport: one JSON request on stdin, one JSON response on stdout.
- Process architecture: x86.
- Database mode: read-only only.
- Raw rows: not returned by the first contract.
- Message bodies: not returned by the first contract.
- DB mutation: never allowed.
Request envelope:
```json
{
"protocol": "isphere.msglib.v1",
"request_id": "example-1",
"op": "self_check",
"args": {}
}
```
Response envelope:
```json
{
"protocol": "isphere.msglib.v1",
"request_id": "example-1",
"op": "self_check",
"ok": true,
"data": {},
"error": null,
"warnings": []
}
```
## Operations
### `self_check`
Returns sidecar identity, protocol, runtime, process bitness, and capability flags.
Key response fields:
- `sidecar_name`
- `sidecar_version`
- `protocol`
- `runtime`
- `process_bits`
- `requires_process_bits`
- `db_mutation_allowed=false`
- `message_body_reads_allowed=false`
### `schema_summary`
Validates provider load and read-only DB open, then returns schema metadata only.
Required args:
- `sqlite_dll_path`: path to bundled `System.Data.SQLite.dll`.
- `db_path`: path to a copied/operator-local `MsgLib.db`.
Optional args:
- `password`: default `123`.
- `include_counts`: default `true`; returns row counts only for target tables.
- `max_tables`: default `128`, clamped to `1..256`.
- `target_tables`: optional list; defaults to known display/message/file metadata tables.
Returned data:
- `metadata_only=true`
- `read_only=true`
- `message_body_values_returned=false`
- `sqlite_version`
- `tables`: table/view names only.
- `columns`: table column names/types only.
- `target_row_counts`: safe row counts for target tables only.
### `display_sources`
Same safe provider/DB validation as `schema_summary`, but additionally maps schema availability to product-facing display source candidates.
Returned data adds:
- `display_sources[]` entries with:
- `source`
- `table`
- `available`
- `required_columns`
- `present_columns`
Initial display sources:
- `contacts_roster` from `TD_Roster`.
- `contacts_effigy` from `TD_CustomEffigy`.
- `recent_display` from `tblRecent`.
- `person_message_names` from `tblPersonMsg`.
- `group_message_names` from `tblMsgGroupPersonMsg`.
- `work_group_auth` from `TD_WorkGroupAuth`.
- `received_file_metadata` from `TD_ReceiveFileRecord`.
- `sent_file_metadata` from `TD_SendFileRecord`.
## Security and scope rules
- The sidecar must open DB files with read-only connection settings.
- `db_path` must point to a file named `MsgLib.db`.
- The first contract returns schema, columns, counts, and display-source availability only.
- It does not return message bodies, contact row values, file paths, or raw row sets.
- It does not implement send, file download, mark-read, login, hooks, injection, or DB writes.
- The Go MCP layer remains the coordinator; this process is a bounded local reader.
## Current implementation
Committed C27 implementation:
- Source: `native/MsgLibReadSidecar/`.
- Build script: `scripts/build-msglib-sidecar.ps1`.
- Verification script: `scripts/verify-msglib-sidecar.ps1`.
- Default verification builds an x86 executable and checks `self_check`.
- Evidence-backed verification with C26 paths checked provider load/read-only DB open and returned 8 display-source candidates.
## Verification
Default verification builds and checks `self_check` only:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1
```
Evidence-backed provider verification is opt-in through environment variables and should use copied DB files only:
```powershell
$env:ISPHERE_MSGLIB_SQLITE_DLL = "<path-to-System.Data.SQLite.dll>"
$env:ISPHERE_MSGLIB_DB = "<path-to-copied-MsgLib.db>"
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1
```

8
docs/source-discovery/capability-source-matrix.md
View File

@@ -17,9 +17,9 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`
| MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice | | MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice |
| --- | --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- | --- |
| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; copied `MsgLib.db` through a future x86 read-only sidecar/helper | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C20 supports and validates the current safe receive-message contract args; C26 validates copied `MsgLib.db` read-only open/schema, but DB-backed message listing is not implemented yet | C27 bounded x86 MsgLib sidecar/helper contract for safe display/schema reads | | `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; copied `MsgLib.db` through a future x86 read-only sidecar/helper | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C20 supports and validates the current safe receive-message contract args; C27 adds a bounded x86 `MsgLibReadSidecar` with schema/display-source operations; DB-backed message listing is still not implemented | C28 Go sidecar client wrapper before MCP/tool integration |
| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md` | C26 validates read-only DB open/schema with the recovered wrapper password; product code still needs a bounded x86 sidecar/helper before DB-backed display names can be exposed | C27 bounded x86 MsgLib sidecar/helper contract | | `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md` | C27 adds and verifies the bounded x86 sidecar/helper contract; Go MCP still needs a sidecar client wrapper before DB-backed display names can be exposed | C28 Go sidecar client wrapper |
| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C26 validates read-only DB open/schema with the recovered wrapper password; product code still needs a bounded x86 sidecar/helper before DB-backed group/member names can be exposed | C27 bounded x86 MsgLib sidecar/helper contract | | `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C27 adds and verifies the bounded x86 sidecar/helper contract; Go MCP still needs a sidecar client wrapper before DB-backed group/member names can be exposed | C28 Go sidecar client wrapper |
| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence | | `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence |
| `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration | | `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration |
| `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated | | `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated |
@@ -39,4 +39,4 @@ Start Stage C with a narrow log-backed `isphere_receive_messages` source abstrac
## Deferred source work ## Deferred source work
`MsgLib.db` has a validated copied read-only open path through the bundled 32-bit `System.Data.SQLite.dll` and password `123`. Because the working provider path is x86 .NET, DB-backed production reads should enter through a bounded sidecar/helper contract rather than direct Go SQLite access. `MsgLib.db` has a validated copied read-only open path through the bundled 32-bit `System.Data.SQLite.dll` and password `123`. C27 adds `MsgLibReadSidecar` as the bounded x86 .NET reader boundary; the next step is a Go sidecar client wrapper and then contact/group display-name integration.

105
docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md
View File

@@ -2106,40 +2106,113 @@ Copied `MsgLib.db` read-only schema extraction result:
**Goal:** Turn the validated C26 read-only DB path into the first production-shaped helper boundary for DB-backed display-name/source metadata, without exposing message bodies or adding write behavior. **Goal:** Turn the validated C26 read-only DB path into the first production-shaped helper boundary for DB-backed display-name/source metadata, without exposing message bodies or adding write behavior.
**Planned files:** **Files:**
- Modify or create docs that define the sidecar/helper JSON contract. - Create: `native/MsgLibReadSidecar/Program.cs`
- Optionally create a minimal committed C# sidecar/helper skeleton if the contract can stay narrow and testable. - Create: `native/MsgLibReadSidecar/SidecarProtocol.cs`
- Modify verification docs/scripts only if an executable smoke is added. - Create: `scripts/build-msglib-sidecar.ps1`
- Create: `scripts/verify-msglib-sidecar.ps1`
- Create: `docs/msglib-read-sidecar-contract.md`
- Modify: `docs/source-discovery/capability-source-matrix.md`
- Modify: `docs/current-status-card.md`
- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`
- Do not commit `System.Data.SQLite.dll`, copied DBs, raw rows, message bodies, personal values, file paths, or generated probe binaries. - Do not commit `System.Data.SQLite.dll`, copied DBs, raw rows, message bodies, personal values, file paths, or generated probe binaries.
**Planned behavior:** **Planned behavior:**
- Treat the DB reader as a dedicated x86 .NET boundary because C26 proved 32-bit SQLite loading is required. - Treat the DB reader as a dedicated x86 .NET boundary because C26 proved 32-bit SQLite loading is required.
- Start with safe operations such as: - Implement safe operations:
- `msglib_self_check`: validates provider load and read-only open against a copied DB path. - `self_check`: sidecar identity, runtime, x86 bitness, and no-mutation flags.
- `msglib_list_display_sources`: returns table/column availability and limited display-name source counts, not raw message content. - `schema_summary`: provider load/read-only DB open plus schema/columns/counts only.
- `display_sources`: schema availability mapped to display-source candidates only.
- Keep Go MCP as the coordinator and use a bounded process/helper contract for Windows/.NET DB access. - Keep Go MCP as the coordinator and use a bounded process/helper contract for Windows/.NET DB access.
- Keep message-body reads, file downloads, sends, writes, hooks, injection, and DB mutation out of scope. - Keep message-body reads, file downloads, sends, writes, hooks, injection, and DB mutation out of scope.
- [ ] **Step 1: Define the sidecar/helper contract** - [x] **Step 1: Define the sidecar/helper contract**
Write the request/response envelope, allowed operations, path rules, limit rules, and redaction rules. Created `docs/msglib-read-sidecar-contract.md` with protocol `isphere.msglib.v1`, request/response envelopes, safe operation contracts, and scope rules.
- [ ] **Step 2: Decide skeleton vs docs-only for this loop** - [x] **Step 2: Implement minimal C# x86 sidecar skeleton**
If the contract is stable enough, create a minimal C# x86 sidecar skeleton with self-check only. Otherwise keep C27 as a contract/design gate and make C28 the skeleton implementation. Created `native/MsgLibReadSidecar` and build/verify scripts. The sidecar compiles x86 and uses reflection to load a caller-supplied `System.Data.SQLite.dll`, so the SQLite binary is not committed.
- [ ] **Step 3: Update status/matrix/plan** - [x] **Step 3: Verify default and evidence-backed smokes**
Record the implementation boundary and next code slice based on actual C27 output. Verified default and evidence-backed sidecar smokes with `scripts\verify-msglib-sidecar.ps1`; evidence-backed smoke used copied DB paths through environment variables and returned `display_source_count=8`.
- [ ] **Step 4: Verify and commit** - [x] **Step 4: Verify and commit**
Run the appropriate docs/code verification before commit: Run final verification before commit:
```powershell ```powershell
git diff --check git diff --check
go test ./... go test ./...
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/verify-go-mcp.ps1 powershell -NoProfile -ExecutionPolicy Bypass -File scripts\build-msglib-sidecar.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1 # with copied DB env paths
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-go-mcp.ps1
```
Final verification passed: `git diff --check`, `go test ./...`, explicit `go build ./cmd/isphere-mcp`, default sidecar smoke, evidence-backed sidecar smoke, and `scripts\verify-go-mcp.ps1`.
---
## Loop C27 Result
Bounded MsgLib sidecar result:
- Added `native/MsgLibReadSidecar` as a dedicated x86 .NET process with protocol `isphere.msglib.v1`.
- Added `self_check`, `schema_summary`, and `display_sources` operations.
- `schema_summary` and `display_sources` require caller-supplied `sqlite_dll_path` and `db_path`, open only files named `MsgLib.db`, set read-only SQLite connection options, and return schema/column/count/display-source metadata only.
- Added `scripts/build-msglib-sidecar.ps1`, compiling with 32-bit .NET Framework `csc.exe` and `/platform:x86`.
- Added `scripts/verify-msglib-sidecar.ps1`, with default self-check verification plus opt-in evidence-backed provider verification through `ISPHERE_MSGLIB_SQLITE_DLL` and `ISPHERE_MSGLIB_DB`.
- Default sidecar verification passed with `process_bits=32`.
- Evidence-backed sidecar verification passed against the copied DB and returned `display_source_count=8`.
- The sidecar is not yet wrapped by Go and is not yet used by MCP tools.
---
## Loop C28: Go `MsgLibReadSidecar` client wrapper
**Goal:** Add a Go-side client wrapper that can call the C27 x86 sidecar through a bounded JSON process contract, without exposing a new MCP tool or reading message bodies yet.
**Planned files:**
- Create: `internal/msglib/sidecar_client.go`
- Create: `internal/msglib/sidecar_client_test.go`
- Optionally update `docs/msglib-read-sidecar-contract.md`
- Modify: `docs/current-status-card.md`
- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`
**Planned behavior:**
- Implement a Go client for `isphere.msglib.v1` with request ids, stdin/stdout JSON, timeout, and structured errors.
- Test with a fake sidecar executable/script first; do not require real `System.Data.SQLite.dll` or real DB in normal unit tests.
- Add env config shape but do not wire MCP tools yet:
- `ISPHERE_MSGLIB_SIDECAR_EXE`
- `ISPHERE_MSGLIB_SQLITE_DLL`
- `ISPHERE_MSGLIB_DB`
- Expose only Go functions for `SelfCheck` and `DisplaySources` metadata.
- Do not add message-body reads, file downloads, sends, writes, hooks, injection, DB mutation, or a public MCP tool in this loop.
- [ ] **Step 1: Write failing fake-sidecar client tests**
Test request envelope, timeout/error handling, and display-source metadata decoding against a fake process.
- [ ] **Step 2: Implement sidecar client**
Use `context.Context`, `exec.CommandContext`, stdin JSON, stdout JSON, stderr capture, and small typed structs.
- [ ] **Step 3: Verify and update docs**
Run Go tests and update status/plan. Keep real evidence-backed sidecar smoke optional through `scripts/verify-msglib-sidecar.ps1`.
- [ ] **Step 4: Full verification and commit**
Run:
```powershell
git diff --check
go test ./...
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\build-msglib-sidecar.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-go-mcp.ps1
``` ```
--- ---

329
native/MsgLibReadSidecar/Program.cs Normal file
View File

@@ -0,0 +1,329 @@
using System;
using System.Collections.Generic;
using System.Data.Common;
using System.IO;
using System.Reflection;
namespace MsgLibReadSidecar
{
internal static class Program
{
private const string Version = "0.1.0";
private const string DefaultPassword = "123";
private static readonly HashSet<string> DefaultTargetTables = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
{
"TD_Roster", "TD_CustomEffigy", "tblRecent", "tblChatLevel", "tblPersonMsg", "tblMsgGroupPersonMsg",
"TD_WorkGroupAuth", "TD_ReceiveFileRecord", "TD_SendFileRecord", "TD_SystemMessageRecord", "TD_MsgReadBurn"
};
[STAThread]
private static int Main(string[] args)
{
string requestText = Console.In.ReadToEnd();
Dictionary<string, object> request;
string requestId = "";
string op = "";
if (!SidecarProtocol.TryParseRequest(requestText, out request, out requestId, out op))
{
Console.WriteLine(SidecarProtocol.ToJson(SidecarProtocol.Failure(requestId, op, "INVALID_REQUEST", "request must be a valid isphere.msglib.v1 JSON object")));
return 0;
}
try
{
Dictionary<string, object> opArgs = SidecarProtocol.GetObject(request, "args");
Dictionary<string, object> response;
switch (op)
{
case "self_check":
response = SidecarProtocol.Success(requestId, op, SelfCheckData());
break;
case "schema_summary":
response = SidecarProtocol.Success(requestId, op, SchemaSummary(opArgs, false));
break;
case "display_sources":
response = SidecarProtocol.Success(requestId, op, SchemaSummary(opArgs, true));
break;
default:
response = SidecarProtocol.Failure(requestId, op, "UNSUPPORTED_OP", "unsupported op: " + op);
break;
}
Console.WriteLine(SidecarProtocol.ToJson(response));
return 0;
}
catch (Exception ex)
{
Console.Error.WriteLine(ex.ToString());
Console.WriteLine(SidecarProtocol.ToJson(SidecarProtocol.Failure(requestId, op, "SIDECAR_FAILED", ex.Message)));
return 0;
}
}
private static Dictionary<string, object> SelfCheckData()
{
return new Dictionary<string, object>
{
{ "sidecar_name", "MsgLibReadSidecar" },
{ "sidecar_version", Version },
{ "protocol", SidecarProtocol.Protocol },
{ "runtime", ".NET Framework " + Environment.Version },
{ "process_bits", IntPtr.Size * 8 },
{ "requires_process_bits", 32 },
{ "db_mutation_allowed", false },
{ "message_body_reads_allowed", false }
};
}
private static Dictionary<string, object> SchemaSummary(Dictionary<string, object> args, bool displayOnly)
{
string sqliteDllPath = SidecarProtocol.GetString(args, "sqlite_dll_path", "");
string dbPath = SidecarProtocol.GetString(args, "db_path", "");
string password = SidecarProtocol.GetString(args, "password", DefaultPassword);
bool includeCounts = SidecarProtocol.GetBool(args, "include_counts", true);
int maxTables = Clamp(SidecarProtocol.GetInt(args, "max_tables", 128), 1, 256);
List<string> requestedTargets = SidecarProtocol.GetStringList(args, "target_tables");
HashSet<string> targetTables = requestedTargets.Count == 0 ? DefaultTargetTables : new HashSet<string>(requestedTargets, StringComparer.OrdinalIgnoreCase);
ValidateInputs(sqliteDllPath, dbPath);
var result = new Dictionary<string, object>
{
{ "metadata_only", true },
{ "read_only", true },
{ "message_body_values_returned", false },
{ "sqlite_dll_path", sqliteDllPath },
{ "db_path", dbPath },
{ "provider", "System.Data.SQLite" },
{ "process_bits", IntPtr.Size * 8 }
};
using (DbConnection connection = OpenSqliteConnection(sqliteDllPath, dbPath, password))
{
connection.Open();
TryExecuteNonQuery(connection, "PRAGMA query_only=ON");
result["sqlite_version"] = ExecuteScalar(connection, "select sqlite_version()");
List<Dictionary<string, object>> tables = ReadMasterTables(connection, maxTables);
List<Dictionary<string, object>> columns = ReadColumns(connection, tables, displayOnly ? targetTables : null);
result["tables"] = tables;
result["columns"] = columns;
result["target_row_counts"] = includeCounts ? ReadTargetCounts(connection, tables, targetTables) : new List<Dictionary<string, object>>();
if (displayOnly)
{
result["display_sources"] = BuildDisplaySources(tables, columns);
}
}
return result;
}
private static void ValidateInputs(string sqliteDllPath, string dbPath)
{
if (string.IsNullOrWhiteSpace(sqliteDllPath) || !File.Exists(sqliteDllPath))
{
throw new ArgumentException("sqlite_dll_path must point to an existing System.Data.SQLite.dll");
}
if (string.IsNullOrWhiteSpace(dbPath) || !File.Exists(dbPath))
{
throw new ArgumentException("db_path must point to an existing copied MsgLib.db");
}
if (!string.Equals(Path.GetFileName(dbPath), "MsgLib.db", StringComparison.OrdinalIgnoreCase))
{
throw new ArgumentException("db_path must point to a copied MsgLib.db file");
}
}
private static DbConnection OpenSqliteConnection(string sqliteDllPath, string dbPath, string password)
{
Assembly assembly = Assembly.LoadFrom(sqliteDllPath);
Type builderType = assembly.GetType("System.Data.SQLite.SQLiteConnectionStringBuilder", true);
object builder = Activator.CreateInstance(builderType);
SetPropertyIfExists(builderType, builder, "DataSource", dbPath);
SetPropertyIfExists(builderType, builder, "Password", password);
SetPropertyIfExists(builderType, builder, "ReadOnly", true);
SetPropertyIfExists(builderType, builder, "Pooling", false);
SetPropertyIfExists(builderType, builder, "FailIfMissing", true);
string connectionString = Convert.ToString(builderType.GetProperty("ConnectionString").GetValue(builder, null));
Type connectionType = assembly.GetType("System.Data.SQLite.SQLiteConnection", true);
object connection = Activator.CreateInstance(connectionType, new object[] { connectionString });
DbConnection dbConnection = connection as DbConnection;
if (dbConnection == null)
{
throw new InvalidOperationException("SQLiteConnection is not a DbConnection");
}
return dbConnection;
}
private static void SetPropertyIfExists(Type type, object target, string name, object value)
{
PropertyInfo property = type.GetProperty(name);
if (property != null && property.CanWrite)
{
property.SetValue(target, value, null);
}
}
private static List<Dictionary<string, object>> ReadMasterTables(DbConnection connection, int maxTables)
{
var rows = new List<Dictionary<string, object>>();
using (DbCommand command = connection.CreateCommand())
{
command.CommandText = "select type, name, tbl_name from sqlite_master where type in ('table','view') order by type, name";
using (DbDataReader reader = command.ExecuteReader())
{
while (reader.Read() && rows.Count < maxTables)
{
rows.Add(new Dictionary<string, object>
{
{ "type", Convert.ToString(reader["type"]) },
{ "name", Convert.ToString(reader["name"]) },
{ "tbl_name", Convert.ToString(reader["tbl_name"]) }
});
}
}
}
return rows;
}
private static List<Dictionary<string, object>> ReadColumns(DbConnection connection, List<Dictionary<string, object>> tables, HashSet<string> onlyTables)
{
var rows = new List<Dictionary<string, object>>();
foreach (Dictionary<string, object> table in tables)
{
string tableName = Convert.ToString(table["name"]);
string tableType = Convert.ToString(table["type"]);
if (!string.Equals(tableType, "table", StringComparison.OrdinalIgnoreCase)) continue;
if (onlyTables != null && !onlyTables.Contains(tableName)) continue;
using (DbCommand command = connection.CreateCommand())
{
command.CommandText = "PRAGMA table_info(" + QuoteIdent(tableName) + ")";
using (DbDataReader reader = command.ExecuteReader())
{
while (reader.Read())
{
rows.Add(new Dictionary<string, object>
{
{ "table", tableName },
{ "cid", Convert.ToString(reader["cid"]) },
{ "name", Convert.ToString(reader["name"]) },
{ "type", Convert.ToString(reader["type"]) },
{ "notnull", Convert.ToString(reader["notnull"]) },
{ "pk", Convert.ToString(reader["pk"]) }
});
}
}
}
}
return rows;
}
private static List<Dictionary<string, object>> ReadTargetCounts(DbConnection connection, List<Dictionary<string, object>> tables, HashSet<string> targetTables)
{
var rows = new List<Dictionary<string, object>>();
foreach (Dictionary<string, object> table in tables)
{
string tableName = Convert.ToString(table["name"]);
string tableType = Convert.ToString(table["type"]);
if (!string.Equals(tableType, "table", StringComparison.OrdinalIgnoreCase)) continue;
if (!targetTables.Contains(tableName)) continue;
rows.Add(new Dictionary<string, object>
{
{ "table", tableName },
{ "row_count", ExecuteScalar(connection, "select count(*) from " + QuoteIdent(tableName)) }
});
}
return rows;
}
private static List<Dictionary<string, object>> BuildDisplaySources(List<Dictionary<string, object>> tables, List<Dictionary<string, object>> columns)
{
var tableSet = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
foreach (Dictionary<string, object> table in tables)
{
if (string.Equals(Convert.ToString(table["type"]), "table", StringComparison.OrdinalIgnoreCase))
{
tableSet.Add(Convert.ToString(table["name"]));
}
}
var columnMap = new Dictionary<string, HashSet<string>>(StringComparer.OrdinalIgnoreCase);
foreach (Dictionary<string, object> column in columns)
{
string table = Convert.ToString(column["table"]);
string name = Convert.ToString(column["name"]);
if (!columnMap.ContainsKey(table)) columnMap[table] = new HashSet<string>(StringComparer.OrdinalIgnoreCase);
columnMap[table].Add(name);
}
var sources = new List<Dictionary<string, object>>();
AddSource(sources, tableSet, columnMap, "contacts_roster", "TD_Roster", new string[] { "JId", "Nick", "GroupName" });
AddSource(sources, tableSet, columnMap, "contacts_effigy", "TD_CustomEffigy", new string[] { "PersonJid", "PersonName" });
AddSource(sources, tableSet, columnMap, "recent_display", "tblRecent", new string[] { "PersonId", "PersonName", "FType" });
AddSource(sources, tableSet, columnMap, "person_message_names", "tblPersonMsg", new string[] { "SndPerson", "SndPersonId", "RecvPerson", "RecvPersonId", "ObjPerson", "ObjPersonId" });
AddSource(sources, tableSet, columnMap, "group_message_names", "tblMsgGroupPersonMsg", new string[] { "MsgGroupId", "MsgGroupName", "SessionPersonId", "SessionPersonName", "GroupType" });
AddSource(sources, tableSet, columnMap, "work_group_auth", "TD_WorkGroupAuth", new string[] { "GroupJID", "GroupName", "ChildGroupJid" });
AddSource(sources, tableSet, columnMap, "received_file_metadata", "TD_ReceiveFileRecord", new string[] { "FileName", "SendPersonJid", "SendPersonName" });
AddSource(sources, tableSet, columnMap, "sent_file_metadata", "TD_SendFileRecord", new string[] { "FileName", "ReceivePersonJid", "ReceivePersonName" });
return sources;
}
private static void AddSource(List<Dictionary<string, object>> sources, HashSet<string> tableSet, Dictionary<string, HashSet<string>> columnMap, string sourceName, string table, string[] columns)
{
var presentColumns = new List<string>();
if (columnMap.ContainsKey(table))
{
foreach (string column in columns)
{
if (columnMap[table].Contains(column)) presentColumns.Add(column);
}
}
sources.Add(new Dictionary<string, object>
{
{ "source", sourceName },
{ "table", table },
{ "available", tableSet.Contains(table) && presentColumns.Count > 0 },
{ "required_columns", columns },
{ "present_columns", presentColumns.ToArray() }
});
}
private static string ExecuteScalar(DbConnection connection, string sql)
{
using (DbCommand command = connection.CreateCommand())
{
command.CommandText = sql;
object value = command.ExecuteScalar();
return Convert.ToString(value);
}
}
private static void TryExecuteNonQuery(DbConnection connection, string sql)
{
try
{
using (DbCommand command = connection.CreateCommand())
{
command.CommandText = sql;
command.ExecuteNonQuery();
}
}
catch
{
}
}
private static string QuoteIdent(string value)
{
return "\"" + value.Replace("\"", "\"\"") + "\"";
}
private static int Clamp(int value, int min, int max)
{
if (value < min) return min;
if (value > max) return max;
return value;
}
}
}

142
native/MsgLibReadSidecar/SidecarProtocol.cs Normal file
View File

@@ -0,0 +1,142 @@
using System;
using System.Collections.Generic;
using System.Web.Script.Serialization;
namespace MsgLibReadSidecar
{
internal static class SidecarProtocol
{
public const string Protocol = "isphere.msglib.v1";
private static readonly JavaScriptSerializer Serializer = new JavaScriptSerializer { MaxJsonLength = 16 * 1024 * 1024, RecursionLimit = 256 };
public static bool TryParseRequest(string text, out Dictionary<string, object> request, out string requestId, out string op)
{
request = null;
requestId = "";
op = "";
if (string.IsNullOrWhiteSpace(text))
{
return false;
}
try
{
object decoded = Serializer.DeserializeObject(text);
request = decoded as Dictionary<string, object>;
if (request == null)
{
return false;
}
string protocol = GetString(request, "protocol", "");
requestId = GetString(request, "request_id", "");
op = GetString(request, "op", "");
return protocol == Protocol && !string.IsNullOrWhiteSpace(op);
}
catch
{
return false;
}
}
public static Dictionary<string, object> Success(string requestId, string op, object data)
{
return Envelope(requestId, op, true, data, null);
}
public static Dictionary<string, object> Failure(string requestId, string op, string code, string message)
{
return Envelope(requestId, op, false, null, new Dictionary<string, object> { { "code", code }, { "message", message } });
}
private static Dictionary<string, object> Envelope(string requestId, string op, bool ok, object data, object error)
{
return new Dictionary<string, object>
{
{ "protocol", Protocol },
{ "request_id", requestId ?? "" },
{ "op", op ?? "" },
{ "ok", ok },
{ "data", data },
{ "error", error },
{ "warnings", new object[0] }
};
}
public static string ToJson(object value)
{
return Serializer.Serialize(value);
}
public static Dictionary<string, object> GetObject(Dictionary<string, object> source, string key)
{
if (source == null || !source.ContainsKey(key) || source[key] == null)
{
return new Dictionary<string, object>();
}
return source[key] as Dictionary<string, object> ?? new Dictionary<string, object>();
}
public static string GetString(Dictionary<string, object> source, string key, string defaultValue)
{
if (source == null || !source.ContainsKey(key) || source[key] == null)
{
return defaultValue;
}
return Convert.ToString(source[key]);
}
public static bool GetBool(Dictionary<string, object> source, string key, bool defaultValue)
{
if (source == null || !source.ContainsKey(key) || source[key] == null)
{
return defaultValue;
}
try { return Convert.ToBoolean(source[key]); }
catch { return defaultValue; }
}
public static int GetInt(Dictionary<string, object> source, string key, int defaultValue)
{
if (source == null || !source.ContainsKey(key) || source[key] == null)
{
return defaultValue;
}
try { return Convert.ToInt32(source[key]); }
catch { return defaultValue; }
}
public static List<string> GetStringList(Dictionary<string, object> source, string key)
{
var values = new List<string>();
if (source == null || !source.ContainsKey(key) || source[key] == null)
{
return values;
}
object raw = source[key];
object[] array = raw as object[];
if (array != null)
{
foreach (object item in array)
{
if (item != null) values.Add(Convert.ToString(item));
}
return values;
}
System.Collections.ArrayList list = raw as System.Collections.ArrayList;
if (list != null)
{
foreach (object item in list)
{
if (item != null) values.Add(Convert.ToString(item));
}
return values;
}
string single = Convert.ToString(raw);
if (!string.IsNullOrWhiteSpace(single)) values.Add(single);
return values;
}
}
}

53
scripts/build-msglib-sidecar.ps1 Normal file
View File

@@ -0,0 +1,53 @@
param(
[string]$OutputDir = "runs/msglib-sidecar",
[string]$Configuration = "Release"
)
$ErrorActionPreference = "Stop"
$repo = Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")
$sourceDir = Join-Path $repo "native/MsgLibReadSidecar"
$outDirPath = Join-Path $repo $OutputDir
$outExe = Join-Path $outDirPath "MsgLibReadSidecar.exe"
if (-not (Test-Path -LiteralPath $sourceDir)) {
throw "source directory not found: $sourceDir"
}
$csc = "$env:WINDIR\Microsoft.NET\Framework\v4.0.30319\csc.exe"
if (-not (Test-Path -LiteralPath $csc)) {
throw "32-bit .NET Framework csc.exe v4.0.30319 not found"
}
New-Item -ItemType Directory -Force -Path $outDirPath | Out-Null
$sources = Get-ChildItem -LiteralPath $sourceDir -Filter '*.cs' | Sort-Object Name | ForEach-Object { $_.FullName }
if (-not $sources) {
throw "no C# sources found in $sourceDir"
}
& $csc `
/nologo `
/target:exe `
/platform:x86 `
/optimize+ `
/warn:0 `
/out:$outExe `
/reference:System.dll `
/reference:System.Core.dll `
/reference:System.Data.dll `
/reference:System.Web.Extensions.dll `
$sources
if ($LASTEXITCODE -ne 0) {
throw "MsgLib sidecar compilation failed with exit code $LASTEXITCODE"
}
[ordered]@{
ok = $true
configuration = $Configuration
platform = "x86"
csc = $csc
source_dir = $sourceDir
output = $outExe
} | ConvertTo-Json -Depth 4 -Compress

81
scripts/verify-msglib-sidecar.ps1 Normal file
View File

@@ -0,0 +1,81 @@
param(
[string]$SidecarExe = "runs/msglib-sidecar/MsgLibReadSidecar.exe",
[string]$SQLiteDllPath = $env:ISPHERE_MSGLIB_SQLITE_DLL,
[string]$DbPath = $env:ISPHERE_MSGLIB_DB,
[switch]$SkipBuild
)
$ErrorActionPreference = "Stop"
$repo = Resolve-Path -LiteralPath (Join-Path $PSScriptRoot "..")
$sidecarPath = Join-Path $repo $SidecarExe
function Invoke-SidecarJson([hashtable]$Request) {
$json = $Request | ConvertTo-Json -Depth 10 -Compress
$output = $json | & $sidecarPath
if ($LASTEXITCODE -ne 0) {
throw "sidecar exited with code $LASTEXITCODE"
}
try {
return $output | ConvertFrom-Json
}
catch {
throw "sidecar output was not JSON: $output"
}
}
if (-not $SkipBuild) {
& powershell -NoProfile -ExecutionPolicy Bypass -File (Join-Path $repo "scripts\build-msglib-sidecar.ps1") | Out-Host
if ($LASTEXITCODE -ne 0) {
throw "build-msglib-sidecar.ps1 failed with exit code $LASTEXITCODE"
}
}
if (-not (Test-Path -LiteralPath $sidecarPath)) {
throw "sidecar not found: $sidecarPath"
}
$self = Invoke-SidecarJson @{
protocol = "isphere.msglib.v1"
request_id = "verify-self-check"
op = "self_check"
args = @{}
}
if (-not $self.ok -or $self.data.sidecar_name -ne "MsgLibReadSidecar" -or $self.data.process_bits -ne 32) {
throw "self_check failed: $($self | ConvertTo-Json -Depth 8 -Compress)"
}
$providerChecked = $false
$displaySources = $null
if ($SQLiteDllPath -and $DbPath) {
if (-not (Test-Path -LiteralPath $SQLiteDllPath)) {
throw "SQLite DLL not found: $SQLiteDllPath"
}
if (-not (Test-Path -LiteralPath $DbPath)) {
throw "MsgLib DB not found: $DbPath"
}
$displaySources = Invoke-SidecarJson @{
protocol = "isphere.msglib.v1"
request_id = "verify-display-sources"
op = "display_sources"
args = @{
sqlite_dll_path = $SQLiteDllPath
db_path = $DbPath
password = "123"
include_counts = $true
max_tables = 64
}
}
if (-not $displaySources.ok -or -not $displaySources.data.metadata_only -or $displaySources.data.message_body_values_returned) {
throw "display_sources failed: $($displaySources | ConvertTo-Json -Depth 12 -Compress)"
}
$providerChecked = $true
}
[ordered]@{
ok = $true
sidecar = (Resolve-Path -LiteralPath $sidecarPath).Path
version = $self.data.sidecar_version
process_bits = $self.data.process_bits
provider_checked = $providerChecked
display_source_count = if ($displaySources) { $displaySources.data.display_sources.Count } else { 0 }
} | ConvertTo-Json -Depth 6 -Compress