feat: add msglib sidecar go client

2026-07-10 02:53:28 +08:00
parent 5b05228ba8
commit 974fcaa018
6 changed files with 544 additions and 17 deletions
--- a/docs/current-status-card.md
+++ b/docs/current-status-card.md
@@ -27,9 +27,9 @@ Remote base before local roadmap commits: `b2d839e Merge branch 'codex/n15-repor
 - `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, eight-tool MCP surface, default empty-source receive/contact/group/file calls, synthetic configured-file receive/contact/group/file smoke, and synthetic configured-directory receive/contact/group/file smoke.
 - `isphere_receive_messages` now returns the contract-facing `ok`, `conversation`, `messages`, `next_cursor`, and `audit` envelope while retaining legacy parser-native message fields for compatibility.
 - `isphere_receive_messages` accepts the read-contract argument set for the current local-readonly path: `conversation_id`, `query`, `since`, `limit`, `include_attachment_metadata`, `source_preference`, `preview`, and empty `cursor`.
- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown until the new x86 `MsgLibReadSidecar` is wrapped from Go and integrated.
+- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown until the x86 `MsgLibReadSidecar` is extended for bounded display-entity reads and integrated.
 - `isphere_receive_files` list mode accepts and validates the safe file-list contract args for the current local-readonly path while keeping download blocked.
- `native/MsgLibReadSidecar` now provides a bounded x86 .NET read-only boundary for `MsgLib.db` schema/display-source checks; it is not yet wired into Go MCP tools.
+- `native/MsgLibReadSidecar` provides a bounded x86 .NET read-only boundary for `MsgLib.db` schema/display-source checks, and `internal/msglib` now wraps it from Go; it is not yet wired into MCP tools and does not return display row values yet.
 - N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures.
 - N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only.

@@ -56,7 +56,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme

 ## Current loop

-Current loop: `Stage C / Loop C28 - Go MsgLib sidecar client wrapper`.
+Current loop: `Stage C / Loop C29 - bounded MsgLib display entity extraction`.

 Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`.

@@ -96,10 +96,11 @@ Current loop output so far:
 25. C25: static .NET wrapper analysis recovered the DAL open path: `HYHC.IMPP.DAL.IMPPDAL` configures `System.Data.SQLite` through `Utilities.Lib.DBSQLite`, sets `DatabaseName` to the DB path and `Password` to `123`, and exposes candidate display/message tables such as `TD_Roster`, `tblRecent`, `tblPersonMsg`, `tblMsgGroupPersonMsg`, and `TD_WorkGroupAuth`.
 26. C26: copied `MsgLib.db` schema extraction succeeded through a 32-bit .NET `System.Data.SQLite` read-only probe with password `123`; all three copied DB candidates opened, with confirmed contact/group/message/file schema tables.
 27. C27: added `native/MsgLibReadSidecar` with protocol `isphere.msglib.v1`, x86 build/verify scripts, and safe `self_check`, `schema_summary`, and `display_sources` operations; evidence-backed smoke returned 8 display-source candidates without row/message values.
+28. C28: added `internal/msglib` Go client wrapper for the sidecar protocol with env config, fake-sidecar unit tests, timeout handling, structured sidecar errors, `SelfCheck`, and `DisplaySources`; no MCP tool wiring or message/display row reads yet.

 ## Next business mainline

-1. Run Stage C Loop C28: add a Go client wrapper for `MsgLibReadSidecar` using env-configured sidecar/SQLite/DB paths, with fake-sidecar tests first; do not expose a new MCP tool or read message bodies yet.
+1. Run Stage C Loop C29: extend the sidecar/client contract for bounded contact/group display-entity extraction only, with tests first; do not expose message bodies, file paths, downloads, sends, writes, hooks, injection, or raw row dumps.
 2. Leave `isphere_send_message` blocked until bridge/API, UIA action-chain, or protocol connector evidence is validated.
 3. Leave actual file download/cache mapping as a later connector node until a message-to-cache hash or client download source is validated.
 4. Keep UIA selector/report work as fallback support, not as the primary roadmap.
--- a/docs/msglib-read-sidecar-contract.md
+++ b/docs/msglib-read-sidecar-contract.md
@@ -129,6 +129,19 @@ Committed C27 implementation:
 - Default verification builds an x86 executable and checks `self_check`.
 - Evidence-backed verification with C26 paths checked provider load/read-only DB open and returned 8 display-source candidates.

+## Go client wrapper
+
+Committed C28 implementation:
+
+- Source: `internal/msglib/`.
+- `ConfigFromEnv` reads `ISPHERE_MSGLIB_SIDECAR_EXE`, `ISPHERE_MSGLIB_SQLITE_DLL`, `ISPHERE_MSGLIB_DB`, and optional `ISPHERE_MSGLIB_PASSWORD`.
+- `NewClient` creates a process-backed client for protocol `isphere.msglib.v1`.
+- `SelfCheck` calls `self_check` and decodes sidecar identity, protocol, bitness, and read-only/no-mutation flags.
+- `DisplaySources` calls `display_sources` and decodes display-source availability metadata.
+- The client uses stdin/stdout JSON, request ids, `context.Context`, per-call timeout, stderr capture, and typed `SidecarError` values.
+- Unit tests use a fake sidecar process; normal tests do not require real `System.Data.SQLite.dll` or copied `MsgLib.db`.
+- C28 does not register a new MCP tool and does not return message bodies, file paths, raw rows, contact row values, or group row values.
+
 ## Verification

 Default verification builds and checks `self_check` only:
--- a/docs/source-discovery/capability-source-matrix.md
+++ b/docs/source-discovery/capability-source-matrix.md
@@ -17,9 +17,9 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`

 | MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice |
 | --- | --- | --- | --- | --- | --- |
-| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; copied `MsgLib.db` through a future x86 read-only sidecar/helper | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C20 supports and validates the current safe receive-message contract args; C27 adds a bounded x86 `MsgLibReadSidecar` with schema/display-source operations; DB-backed message listing is still not implemented | C28 Go sidecar client wrapper before MCP/tool integration |
-| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md` | C27 adds and verifies the bounded x86 sidecar/helper contract; Go MCP still needs a sidecar client wrapper before DB-backed display names can be exposed | C28 Go sidecar client wrapper |
-| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C27 adds and verifies the bounded x86 sidecar/helper contract; Go MCP still needs a sidecar client wrapper before DB-backed group/member names can be exposed | C28 Go sidecar client wrapper |
+| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; copied `MsgLib.db` through a future x86 read-only sidecar/helper | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C20 supports and validates the current safe receive-message contract args; C28 adds the Go `internal/msglib` sidecar wrapper for schema/display-source operations; DB-backed message listing is still not implemented | Defer DB-backed message listing; first run C29 bounded contact/group display-entity extraction |
+| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md` | C28 adds the Go sidecar client wrapper; the sidecar still needs a bounded display-entity operation before DB-backed display names can be exposed | C29 bounded contact display-entity extraction |
+| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C28 adds the Go sidecar client wrapper; the sidecar still needs a bounded display-entity operation before DB-backed group/member names can be exposed | C29 bounded group display-entity extraction |
 | `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\<hash>` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence |
 | `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration |
 | `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated |
@@ -39,4 +39,4 @@ Start Stage C with a narrow log-backed `isphere_receive_messages` source abstrac

 ## Deferred source work

-`MsgLib.db` has a validated copied read-only open path through the bundled 32-bit `System.Data.SQLite.dll` and password `123`. C27 adds `MsgLibReadSidecar` as the bounded x86 .NET reader boundary; the next step is a Go sidecar client wrapper and then contact/group display-name integration.
+`MsgLib.db` has a validated copied read-only open path through the bundled 32-bit `System.Data.SQLite.dll` and password `123`. C27 adds `MsgLibReadSidecar` as the bounded x86 .NET reader boundary, and C28 adds the Go `internal/msglib` process client. The next step is a bounded display-entity operation before contact/group display-name integration.
--- a/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md
+++ b/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md
@@ -2174,10 +2174,11 @@ Bounded MsgLib sidecar result:

 **Goal:** Add a Go-side client wrapper that can call the C27 x86 sidecar through a bounded JSON process contract, without exposing a new MCP tool or reading message bodies yet.

-**Planned files:**
+**Files:**
 - Create: `internal/msglib/sidecar_client.go`
 - Create: `internal/msglib/sidecar_client_test.go`
- Optionally update `docs/msglib-read-sidecar-contract.md`
+- Modify: `docs/msglib-read-sidecar-contract.md`
+- Modify: `docs/source-discovery/capability-source-matrix.md`
 - Modify: `docs/current-status-card.md`
 - Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`

@@ -2191,25 +2192,89 @@ Bounded MsgLib sidecar result:
 - Expose only Go functions for `SelfCheck` and `DisplaySources` metadata.
 - Do not add message-body reads, file downloads, sends, writes, hooks, injection, DB mutation, or a public MCP tool in this loop.

- [ ] **Step 1: Write failing fake-sidecar client tests**
+- [x] **Step 1: Write failing fake-sidecar client tests**

-Test request envelope, timeout/error handling, and display-source metadata decoding against a fake process.
+Created `internal/msglib/sidecar_client_test.go` with fake process tests for request envelope, `self_check`, `display_sources`, structured sidecar errors, timeout handling, and env config. Verified RED with undefined `Config`, `Client`, `Protocol`, and `SidecarError` symbols.

- [ ] **Step 2: Implement sidecar client**
+- [x] **Step 2: Implement sidecar client**

-Use `context.Context`, `exec.CommandContext`, stdin JSON, stdout JSON, stderr capture, and small typed structs.
+Created `internal/msglib/sidecar_client.go` with protocol constants, `ConfigFromEnv`, `NewClient`, `SelfCheck`, `DisplaySources`, request ids, `exec.CommandContext`, stdin/stdout JSON, timeout, stderr capture, response validation, and typed `SidecarError` handling.

- [ ] **Step 3: Verify and update docs**
+- [x] **Step 3: Verify and update docs**

-Run Go tests and update status/plan. Keep real evidence-backed sidecar smoke optional through `scripts/verify-msglib-sidecar.ps1`.
+Ran package tests for `internal/msglib` and updated the sidecar contract, capability matrix, current status card, and this plan. Real evidence-backed sidecar smoke remains covered by `scripts\verify-msglib-sidecar.ps1` rather than normal Go unit tests.

- [ ] **Step 4: Full verification and commit**
+- [x] **Step 4: Full verification and commit**
+
+Run before commit:
+
+```powershell
+git diff --check
+go test ./...
+go build ./cmd/isphere-mcp
+powershell -NoProfile -ExecutionPolicy Bypass -File scripts\build-msglib-sidecar.ps1
+powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1
+powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1 # with copied DB env paths
+powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-go-mcp.ps1
+```
+
+Final verification passed: `git diff --check`, `go test ./...`, explicit `go build ./cmd/isphere-mcp`, default sidecar smoke, evidence-backed sidecar smoke, and `scripts\verify-go-mcp.ps1`.
+
+---
+
+## Loop C28 Result
+
+Go sidecar client result:
+
+- Added `internal/msglib` as the internal Go process client for `isphere.msglib.v1`.
+- Added fake-sidecar unit coverage for envelope validation, `SelfCheck`, `DisplaySources`, structured `SidecarError`, timeout behavior, and env config.
+- `ConfigFromEnv` reads `ISPHERE_MSGLIB_SIDECAR_EXE`, `ISPHERE_MSGLIB_SQLITE_DLL`, `ISPHERE_MSGLIB_DB`, and optional `ISPHERE_MSGLIB_PASSWORD`.
+- C28 still does not register a new MCP tool, does not read message bodies, does not return contact/group row values, and does not implement file download/send/write behavior.
+- Next gap: C29 must add a bounded display-entity operation before DB-backed contact/group display names can be exposed.
+
+---
+
+## Loop C29: Bounded `MsgLib.db` display entity extraction
+
+**Goal:** Extend the sidecar/client boundary from schema/display-source availability to bounded contact/group display-entity metadata, so a later loop can enrich `isphere_search_contacts` and `isphere_search_groups` without returning raw rows, message bodies, file paths, downloads, sends, writes, hooks, injection, or DB mutations.
+
+**Planned files:**
+- Modify: `native/MsgLibReadSidecar/Program.cs`
+- Modify: `docs/msglib-read-sidecar-contract.md`
+- Modify: `internal/msglib/sidecar_client.go`
+- Modify: `internal/msglib/sidecar_client_test.go`
+- Modify: `scripts/verify-msglib-sidecar.ps1`
+- Modify: `docs/current-status-card.md`
+- Modify: `docs/source-discovery/capability-source-matrix.md`
+- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`
+
+**Planned behavior:**
+- Add a sidecar operation tentatively named `display_entities`.
+- Request args: `sqlite_dll_path`, `db_path`, optional `password`, `entity_type` (`contacts` or `groups`), optional `query`, and bounded `limit`.
+- Response data: normalized metadata only, such as `entity_type`, `source_table`, `jid`, `display_name`, `confidence`, and `matched_columns`.
+- Never return message bodies, file paths, raw SQL row maps, attachment contents, send/write capabilities, or unbounded result sets.
+- Use tests first. Normal Go tests should still use fake sidecar data; real copied DB verification stays opt-in through script/env paths.
+
+- [ ] **Step 1: Write failing display-entity contract tests**
+
+Add sidecar/client tests for bounded contact/group entity decoding, query/limit validation, and rejection of unsupported entity types.
+
+- [ ] **Step 2: Implement sidecar `display_entities` operation**
+
+Implement allowlisted table/column reads only for contact/group identity metadata, with parameterized query/limit and no raw row dumps.
+
+- [ ] **Step 3: Extend Go client and verification script**
+
+Add Go wrapper types/methods and optional evidence-backed verification that reports only counts/source-table names, not personal values.
+
+- [ ] **Step 4: Update docs, run full verification, and commit**

 Run:

 ```powershell
 git diff --check
 go test ./...
+go build ./cmd/isphere-mcp
 powershell -NoProfile -ExecutionPolicy Bypass -File scripts\build-msglib-sidecar.ps1
 powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1
 powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-go-mcp.ps1
--- a/internal/msglib/sidecar_client.go
+++ b/internal/msglib/sidecar_client.go
@@ -0,0 +1,231 @@
+package msglib
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"os"
+	"os/exec"
+	"strings"
+	"sync/atomic"
+	"time"
+)
+
+const (
+	Protocol        = "isphere.msglib.v1"
+	DefaultPassword = "123"
+)
+
+var requestCounter uint64
+
+// Config describes the local sidecar process and copied MsgLib DB inputs.
+type Config struct {
+	SidecarPath   string
+	SidecarArgs   []string
+	SQLiteDLLPath string
+	DBPath        string
+	Password      string
+	Timeout       time.Duration
+	ExtraEnv      []string
+}
+
+// ConfigFromEnv reads the C28 environment shape without wiring it into MCP tools.
+func ConfigFromEnv() (Config, error) {
+	cfg := Config{
+		SidecarPath:   strings.TrimSpace(os.Getenv("ISPHERE_MSGLIB_SIDECAR_EXE")),
+		SQLiteDLLPath: strings.TrimSpace(os.Getenv("ISPHERE_MSGLIB_SQLITE_DLL")),
+		DBPath:        strings.TrimSpace(os.Getenv("ISPHERE_MSGLIB_DB")),
+		Password:      strings.TrimSpace(os.Getenv("ISPHERE_MSGLIB_PASSWORD")),
+	}
+	if cfg.Password == "" {
+		cfg.Password = DefaultPassword
+	}
+	if cfg.SidecarPath == "" {
+		return cfg, errors.New("ISPHERE_MSGLIB_SIDECAR_EXE is not configured")
+	}
+	return cfg, nil
+}
+
+// NewClient returns a process-backed MsgLib sidecar client.
+func NewClient(cfg Config) *Client {
+	if cfg.Password == "" {
+		cfg.Password = DefaultPassword
+	}
+	if cfg.Timeout == 0 {
+		cfg.Timeout = 10 * time.Second
+	}
+	return &Client{config: cfg}
+}
+
+type Client struct {
+	config Config
+}
+
+type SelfCheckResult struct {
+	SidecarName    string `json:"sidecar_name"`
+	SidecarVersion string `json:"sidecar_version"`
+	Protocol       string `json:"protocol"`
+	Runtime        string `json:"runtime"`
+	ProcessBits    int    `json:"process_bits"`
+	ReadOnly       bool   `json:"read_only"`
+	MutatesDB      bool   `json:"mutates_db"`
+}
+
+type DisplaySource struct {
+	Source          string   `json:"source"`
+	Table           string   `json:"table"`
+	Available       bool     `json:"available"`
+	RequiredColumns []string `json:"required_columns"`
+	PresentColumns  []string `json:"present_columns"`
+}
+
+type SidecarError struct {
+	Code    string `json:"code"`
+	Message string `json:"message"`
+}
+
+func (e *SidecarError) Error() string {
+	if e == nil {
+		return "msglib sidecar error"
+	}
+	if e.Code == "" {
+		return "msglib sidecar error: " + e.Message
+	}
+	if e.Message == "" {
+		return "msglib sidecar error: " + e.Code
+	}
+	return fmt.Sprintf("msglib sidecar error %s: %s", e.Code, e.Message)
+}
+
+func (c *Client) SelfCheck(ctx context.Context) (SelfCheckResult, error) {
+	var out SelfCheckResult
+	err := c.call(ctx, "self_check", map[string]any{}, &out)
+	return out, err
+}
+
+func (c *Client) DisplaySources(ctx context.Context) ([]DisplaySource, error) {
+	args := map[string]any{
+		"sqlite_dll_path": c.config.SQLiteDLLPath,
+		"db_path":         c.config.DBPath,
+		"password":        c.config.Password,
+	}
+	var out displaySourcesData
+	if err := c.call(ctx, "display_sources", args, &out); err != nil {
+		return nil, err
+	}
+	return out.DisplaySources, nil
+}
+
+type displaySourcesData struct {
+	DisplaySources []DisplaySource `json:"display_sources"`
+}
+
+type requestEnvelope struct {
+	Protocol  string         `json:"protocol"`
+	RequestID string         `json:"request_id"`
+	Op        string         `json:"op"`
+	Args      map[string]any `json:"args,omitempty"`
+}
+
+type responseEnvelope struct {
+	Protocol  string          `json:"protocol"`
+	RequestID string          `json:"request_id"`
+	Op        string          `json:"op"`
+	OK        bool            `json:"ok"`
+	Data      json.RawMessage `json:"data"`
+	Error     *SidecarError   `json:"error"`
+	Warnings  []string        `json:"warnings"`
+}
+
+func (c *Client) call(ctx context.Context, op string, args map[string]any, out any) error {
+	if ctx == nil {
+		ctx = context.Background()
+	}
+	if strings.TrimSpace(c.config.SidecarPath) == "" {
+		return errors.New("msglib sidecar path is not configured")
+	}
+
+	requestID := nextRequestID()
+	req := requestEnvelope{
+		Protocol:  Protocol,
+		RequestID: requestID,
+		Op:        op,
+		Args:      args,
+	}
+	payload, err := json.Marshal(req)
+	if err != nil {
+		return fmt.Errorf("marshal msglib sidecar request: %w", err)
+	}
+	payload = append(payload, '\n')
+
+	callCtx := ctx
+	var cancel context.CancelFunc
+	if c.config.Timeout > 0 {
+		callCtx, cancel = context.WithTimeout(ctx, c.config.Timeout)
+		defer cancel()
+	}
+
+	cmd := exec.CommandContext(callCtx, c.config.SidecarPath, c.config.SidecarArgs...)
+	cmd.Stdin = bytes.NewReader(payload)
+	var stdout bytes.Buffer
+	var stderr bytes.Buffer
+	cmd.Stdout = &stdout
+	cmd.Stderr = &stderr
+	if len(c.config.ExtraEnv) > 0 {
+		cmd.Env = append(os.Environ(), c.config.ExtraEnv...)
+	}
+
+	runErr := cmd.Run()
+	if callCtx.Err() != nil {
+		if errors.Is(callCtx.Err(), context.DeadlineExceeded) {
+			return fmt.Errorf("msglib sidecar %s timed out: %w", op, context.DeadlineExceeded)
+		}
+		return fmt.Errorf("msglib sidecar %s context failed: %w", op, callCtx.Err())
+	}
+	if runErr != nil {
+		stderrText := strings.TrimSpace(stderr.String())
+		if stderrText == "" {
+			return fmt.Errorf("msglib sidecar %s process failed: %w", op, runErr)
+		}
+		return fmt.Errorf("msglib sidecar %s process failed: %w: %s", op, runErr, stderrText)
+	}
+
+	body := bytes.TrimSpace(stdout.Bytes())
+	if len(body) == 0 {
+		return fmt.Errorf("msglib sidecar %s returned empty stdout", op)
+	}
+
+	var resp responseEnvelope
+	if err := json.Unmarshal(body, &resp); err != nil {
+		return fmt.Errorf("decode msglib sidecar %s response: %w", op, err)
+	}
+	if resp.Protocol != Protocol {
+		return fmt.Errorf("msglib sidecar %s response protocol = %q, want %q", op, resp.Protocol, Protocol)
+	}
+	if resp.RequestID != requestID {
+		return fmt.Errorf("msglib sidecar %s response request_id = %q, want %q", op, resp.RequestID, requestID)
+	}
+	if resp.Op != op {
+		return fmt.Errorf("msglib sidecar response op = %q, want %q", resp.Op, op)
+	}
+	if !resp.OK {
+		if resp.Error != nil {
+			return fmt.Errorf("msglib sidecar %s failed: %w", op, resp.Error)
+		}
+		return fmt.Errorf("msglib sidecar %s failed", op)
+	}
+	if out == nil || len(resp.Data) == 0 || bytes.Equal(resp.Data, []byte("null")) {
+		return nil
+	}
+	if err := json.Unmarshal(resp.Data, out); err != nil {
+		return fmt.Errorf("decode msglib sidecar %s data: %w", op, err)
+	}
+	return nil
+}
+
+func nextRequestID() string {
+	n := atomic.AddUint64(&requestCounter, 1)
+	return fmt.Sprintf("go-%d-%d", time.Now().UnixNano(), n)
+}
--- a/internal/msglib/sidecar_client_test.go
+++ b/internal/msglib/sidecar_client_test.go
@@ -0,0 +1,217 @@
+package msglib
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"os"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestSelfCheckSendsProtocolEnvelopeAndDecodesResponse(t *testing.T) {
+	client := newFakeClient(t, "self_check_ok", Config{})
+
+	got, err := client.SelfCheck(context.Background())
+	if err != nil {
+		t.Fatalf("SelfCheck returned error: %v", err)
+	}
+	if got.SidecarName != "MsgLibReadSidecar" {
+		t.Fatalf("SidecarName = %q, want MsgLibReadSidecar", got.SidecarName)
+	}
+	if got.Protocol != Protocol {
+		t.Fatalf("Protocol = %q, want %q", got.Protocol, Protocol)
+	}
+	if got.ProcessBits != 32 || !got.ReadOnly || got.MutatesDB {
+		t.Fatalf("unexpected safety fields: bits=%d read_only=%v mutates_db=%v", got.ProcessBits, got.ReadOnly, got.MutatesDB)
+	}
+}
+
+func TestDisplaySourcesPassesConfiguredPathsAndDecodesMetadata(t *testing.T) {
+	cfg := Config{
+		SQLiteDLLPath: `C:\fixture\System.Data.SQLite.dll`,
+		DBPath:        `C:\fixture\MsgLib.db`,
+		Password:      "123",
+	}
+	client := newFakeClient(t, "display_sources_ok", cfg)
+
+	got, err := client.DisplaySources(context.Background())
+	if err != nil {
+		t.Fatalf("DisplaySources returned error: %v", err)
+	}
+	if len(got) != 2 {
+		t.Fatalf("len(DisplaySources) = %d, want 2", len(got))
+	}
+	if got[0].Source != "contacts_roster" || got[0].Table != "TD_Roster" || !got[0].Available {
+		t.Fatalf("unexpected first display source: %+v", got[0])
+	}
+	if strings.Join(got[0].RequiredColumns, ",") != "UserJID,Name" {
+		t.Fatalf("required columns = %#v", got[0].RequiredColumns)
+	}
+	if got[1].Source != "work_group_auth" || got[1].Available {
+		t.Fatalf("unexpected second display source: %+v", got[1])
+	}
+}
+
+func TestClientReturnsStructuredSidecarError(t *testing.T) {
+	client := newFakeClient(t, "sidecar_error", Config{})
+
+	_, err := client.SelfCheck(context.Background())
+	if err == nil {
+		t.Fatal("SelfCheck returned nil error, want sidecar error")
+	}
+	var sidecarErr *SidecarError
+	if !errors.As(err, &sidecarErr) {
+		t.Fatalf("error %T %[1]v does not wrap *SidecarError", err)
+	}
+	if sidecarErr.Code != "DB_OPEN_FAILED" || !strings.Contains(sidecarErr.Message, "read-only open failed") {
+		t.Fatalf("unexpected sidecar error: %+v", sidecarErr)
+	}
+}
+
+func TestClientTimeoutStopsSidecar(t *testing.T) {
+	client := newFakeClient(t, "hang", Config{Timeout: 150 * time.Millisecond})
+
+	_, err := client.SelfCheck(context.Background())
+	if err == nil {
+		t.Fatal("SelfCheck returned nil error, want timeout")
+	}
+	if !errors.Is(err, context.DeadlineExceeded) && !strings.Contains(strings.ToLower(err.Error()), "deadline") {
+		t.Fatalf("SelfCheck error = %v, want deadline exceeded", err)
+	}
+}
+
+func TestConfigFromEnvReadsSidecarShape(t *testing.T) {
+	t.Setenv("ISPHERE_MSGLIB_SIDECAR_EXE", `C:\tools\MsgLibReadSidecar.exe`)
+	t.Setenv("ISPHERE_MSGLIB_SQLITE_DLL", `C:\tools\System.Data.SQLite.dll`)
+	t.Setenv("ISPHERE_MSGLIB_DB", `C:\copy\MsgLib.db`)
+
+	cfg, err := ConfigFromEnv()
+	if err != nil {
+		t.Fatalf("ConfigFromEnv returned error: %v", err)
+	}
+	if cfg.SidecarPath != `C:\tools\MsgLibReadSidecar.exe` {
+		t.Fatalf("SidecarPath = %q", cfg.SidecarPath)
+	}
+	if cfg.SQLiteDLLPath != `C:\tools\System.Data.SQLite.dll` || cfg.DBPath != `C:\copy\MsgLib.db` {
+		t.Fatalf("unexpected DB config: %+v", cfg)
+	}
+	if cfg.Password != "123" {
+		t.Fatalf("Password = %q, want default 123", cfg.Password)
+	}
+}
+
+func newFakeClient(t *testing.T, mode string, overrides Config) *Client {
+	t.Helper()
+	exe, err := os.Executable()
+	if err != nil {
+		t.Fatalf("os.Executable: %v", err)
+	}
+	cfg := overrides
+	cfg.SidecarPath = exe
+	cfg.SidecarArgs = []string{"-test.run=TestFakeSidecarProcess", "--"}
+	cfg.ExtraEnv = append(cfg.ExtraEnv, "ISPHERE_MSGLIB_FAKE_SIDECAR=1", "ISPHERE_MSGLIB_FAKE_MODE="+mode)
+	if cfg.Timeout == 0 {
+		cfg.Timeout = 2 * time.Second
+	}
+	return NewClient(cfg)
+}
+
+func TestFakeSidecarProcess(t *testing.T) {
+	if os.Getenv("ISPHERE_MSGLIB_FAKE_SIDECAR") != "1" {
+		return
+	}
+	os.Exit(fakeSidecarMain())
+}
+
+func fakeSidecarMain() int {
+	mode := os.Getenv("ISPHERE_MSGLIB_FAKE_MODE")
+	if mode == "hang" {
+		time.Sleep(5 * time.Second)
+		return 0
+	}
+
+	var req map[string]any
+	if err := json.NewDecoder(os.Stdin).Decode(&req); err != nil {
+		fmt.Fprintf(os.Stderr, "decode request: %v\n", err)
+		return 2
+	}
+	protocol, _ := req["protocol"].(string)
+	op, _ := req["op"].(string)
+	requestID, _ := req["request_id"].(string)
+	if protocol != Protocol || op == "" || requestID == "" {
+		fmt.Fprintf(os.Stderr, "bad envelope: protocol=%q op=%q request_id=%q\n", protocol, op, requestID)
+		return 2
+	}
+
+	if mode == "sidecar_error" {
+		return writeFakeResponse(map[string]any{
+			"protocol":   Protocol,
+			"request_id": requestID,
+			"op":         op,
+			"ok":         false,
+			"data":       nil,
+			"error": map[string]any{
+				"code":    "DB_OPEN_FAILED",
+				"message": "read-only open failed",
+			},
+			"warnings": []any{},
+		})
+	}
+
+	switch op {
+	case "self_check":
+		return writeFakeResponse(map[string]any{
+			"protocol":   Protocol,
+			"request_id": requestID,
+			"op":         op,
+			"ok":         true,
+			"data": map[string]any{
+				"sidecar_name":    "MsgLibReadSidecar",
+				"sidecar_version": "0.1.0",
+				"protocol":        Protocol,
+				"process_bits":    32,
+				"read_only":       true,
+				"mutates_db":      false,
+			},
+			"error":    nil,
+			"warnings": []any{},
+		})
+	case "display_sources":
+		args, _ := req["args"].(map[string]any)
+		if args["sqlite_dll_path"] != `C:\fixture\System.Data.SQLite.dll` || args["db_path"] != `C:\fixture\MsgLib.db` || args["password"] != "123" {
+			fmt.Fprintf(os.Stderr, "bad display_sources args: %#v\n", args)
+			return 2
+		}
+		return writeFakeResponse(map[string]any{
+			"protocol":   Protocol,
+			"request_id": requestID,
+			"op":         op,
+			"ok":         true,
+			"data": map[string]any{
+				"metadata_only":                true,
+				"read_only":                    true,
+				"message_body_values_returned": false,
+				"display_sources": []any{
+					map[string]any{"source": "contacts_roster", "table": "TD_Roster", "available": true, "required_columns": []any{"UserJID", "Name"}, "present_columns": []any{"UserJID", "Name", "Mobile"}},
+					map[string]any{"source": "work_group_auth", "table": "TD_WorkGroupAuth", "available": false, "required_columns": []any{"GroupJID", "GroupName"}, "present_columns": []any{"GroupJID"}},
+				},
+			},
+			"error":    nil,
+			"warnings": []any{},
+		})
+	default:
+		fmt.Fprintf(os.Stderr, "unexpected op: %s\n", op)
+		return 2
+	}
+}
+
+func writeFakeResponse(resp map[string]any) int {
+	if err := json.NewEncoder(os.Stdout).Encode(resp); err != nil {
+		fmt.Fprintf(os.Stderr, "encode response: %v\n", err)
+		return 2
+	}
+	return 0
+}