diff --git a/docs/current-status-card.md b/docs/current-status-card.md index 3bcf671..93f58b2 100644 --- a/docs/current-status-card.md +++ b/docs/current-status-card.md @@ -27,9 +27,9 @@ Remote base before local roadmap commits: `b2d839e Merge branch 'codex/n15-repor - `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, eight-tool MCP surface, default empty-source receive/contact/group/file calls, synthetic configured-file receive/contact/group/file smoke, and synthetic configured-directory receive/contact/group/file smoke. - `isphere_receive_messages` now returns the contract-facing `ok`, `conversation`, `messages`, `next_cursor`, and `audit` envelope while retaining legacy parser-native message fields for compatibility. - `isphere_receive_messages` accepts the read-contract argument set for the current local-readonly path: `conversation_id`, `query`, `since`, `limit`, `include_attachment_metadata`, `source_preference`, `preview`, and empty `cursor`. -- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown until the new x86 `MsgLibReadSidecar` is wrapped from Go and integrated. +- `isphere_search_contacts` and `isphere_search_groups` accept and validate the safe search-contract args for the current local-readonly path, but display names/member data remain JID-derived/unknown until the x86 `MsgLibReadSidecar` is extended for bounded display-entity reads and integrated. - `isphere_receive_files` list mode accepts and validates the safe file-list contract args for the current local-readonly path while keeping download blocked. -- `native/MsgLibReadSidecar` now provides a bounded x86 .NET read-only boundary for `MsgLib.db` schema/display-source checks; it is not yet wired into Go MCP tools. +- `native/MsgLibReadSidecar` provides a bounded x86 .NET read-only boundary for `MsgLib.db` schema/display-source checks, and `internal/msglib` now wraps it from Go; it is not yet wired into MCP tools and does not return display row values yet. - N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures. - N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only. @@ -56,7 +56,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme ## Current loop -Current loop: `Stage C / Loop C28 - Go MsgLib sidecar client wrapper`. +Current loop: `Stage C / Loop C29 - bounded MsgLib display entity extraction`. Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`. @@ -96,10 +96,11 @@ Current loop output so far: 25. C25: static .NET wrapper analysis recovered the DAL open path: `HYHC.IMPP.DAL.IMPPDAL` configures `System.Data.SQLite` through `Utilities.Lib.DBSQLite`, sets `DatabaseName` to the DB path and `Password` to `123`, and exposes candidate display/message tables such as `TD_Roster`, `tblRecent`, `tblPersonMsg`, `tblMsgGroupPersonMsg`, and `TD_WorkGroupAuth`. 26. C26: copied `MsgLib.db` schema extraction succeeded through a 32-bit .NET `System.Data.SQLite` read-only probe with password `123`; all three copied DB candidates opened, with confirmed contact/group/message/file schema tables. 27. C27: added `native/MsgLibReadSidecar` with protocol `isphere.msglib.v1`, x86 build/verify scripts, and safe `self_check`, `schema_summary`, and `display_sources` operations; evidence-backed smoke returned 8 display-source candidates without row/message values. +28. C28: added `internal/msglib` Go client wrapper for the sidecar protocol with env config, fake-sidecar unit tests, timeout handling, structured sidecar errors, `SelfCheck`, and `DisplaySources`; no MCP tool wiring or message/display row reads yet. ## Next business mainline -1. Run Stage C Loop C28: add a Go client wrapper for `MsgLibReadSidecar` using env-configured sidecar/SQLite/DB paths, with fake-sidecar tests first; do not expose a new MCP tool or read message bodies yet. +1. Run Stage C Loop C29: extend the sidecar/client contract for bounded contact/group display-entity extraction only, with tests first; do not expose message bodies, file paths, downloads, sends, writes, hooks, injection, or raw row dumps. 2. Leave `isphere_send_message` blocked until bridge/API, UIA action-chain, or protocol connector evidence is validated. 3. Leave actual file download/cache mapping as a later connector node until a message-to-cache hash or client download source is validated. 4. Keep UIA selector/report work as fallback support, not as the primary roadmap. diff --git a/docs/msglib-read-sidecar-contract.md b/docs/msglib-read-sidecar-contract.md index 6abe77f..9636129 100644 --- a/docs/msglib-read-sidecar-contract.md +++ b/docs/msglib-read-sidecar-contract.md @@ -129,6 +129,19 @@ Committed C27 implementation: - Default verification builds an x86 executable and checks `self_check`. - Evidence-backed verification with C26 paths checked provider load/read-only DB open and returned 8 display-source candidates. +## Go client wrapper + +Committed C28 implementation: + +- Source: `internal/msglib/`. +- `ConfigFromEnv` reads `ISPHERE_MSGLIB_SIDECAR_EXE`, `ISPHERE_MSGLIB_SQLITE_DLL`, `ISPHERE_MSGLIB_DB`, and optional `ISPHERE_MSGLIB_PASSWORD`. +- `NewClient` creates a process-backed client for protocol `isphere.msglib.v1`. +- `SelfCheck` calls `self_check` and decodes sidecar identity, protocol, bitness, and read-only/no-mutation flags. +- `DisplaySources` calls `display_sources` and decodes display-source availability metadata. +- The client uses stdin/stdout JSON, request ids, `context.Context`, per-call timeout, stderr capture, and typed `SidecarError` values. +- Unit tests use a fake sidecar process; normal tests do not require real `System.Data.SQLite.dll` or copied `MsgLib.db`. +- C28 does not register a new MCP tool and does not return message bodies, file paths, raw rows, contact row values, or group row values. + ## Verification Default verification builds and checks `self_check` only: diff --git a/docs/source-discovery/capability-source-matrix.md b/docs/source-discovery/capability-source-matrix.md index 68f545f..1ece41e 100644 --- a/docs/source-discovery/capability-source-matrix.md +++ b/docs/source-discovery/capability-source-matrix.md @@ -17,9 +17,9 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md` | MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice | | --- | --- | --- | --- | --- | --- | -| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; copied `MsgLib.db` through a future x86 read-only sidecar/helper | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C20 supports and validates the current safe receive-message contract args; C27 adds a bounded x86 `MsgLibReadSidecar` with schema/display-source operations; DB-backed message listing is still not implemented | C28 Go sidecar client wrapper before MCP/tool integration | -| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md` | C27 adds and verifies the bounded x86 sidecar/helper contract; Go MCP still needs a sidecar client wrapper before DB-backed display names can be exposed | C28 Go sidecar client wrapper | -| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C27 adds and verifies the bounded x86 sidecar/helper contract; Go MCP still needs a sidecar client wrapper before DB-backed group/member names can be exposed | C28 Go sidecar client wrapper | +| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `` stanzas via `internal/isphere.EncryptedPacketLogSource`; configured by `ISPHERE_PACKET_LOG_FILE` or `ISPHERE_PACKET_LOG_DIR` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; copied `MsgLib.db` through a future x86 read-only sidecar/helper | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence`; `docs/source-discovery/2026-07-10-dotnet-wrapper-static-analysis.md` | C20 supports and validates the current safe receive-message contract args; C28 adds the Go `internal/msglib` sidecar wrapper for schema/display-source operations; DB-backed message listing is still not implemented | Defer DB-backed message listing; first run C29 bounded contact/group display-entity extraction | +| `isphere_search_contacts` | bare sender/receiver JIDs extracted from normalized log-backed messages loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `TD_Roster`, `TD_CustomEffigy`, `tblRecent`, `tblChatLevel`, `tblPersonMsg`; decrypted roster/contact stanzas from `Smark.SendReceive`; UIA helper source if display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md` | C28 adds the Go sidecar client wrapper; the sidecar still needs a bounded display-entity operation before DB-backed display names can be exposed | C29 bounded contact display-entity extraction | +| `isphere_search_groups` | decrypted `PacketReader.ProcessPacket` `type="groupchat"` stanzas and conference/MUC JIDs loaded from configured PacketReader file or directory source | validated copied `MsgLib.db` tables: `tblMsgGroupPersonMsg`, `TD_WorkGroupAuth`, `tblRecent`; UIA helper source if group display names are still missing | `docs/source-discovery/2026-07-10-msglib-readonly-schema-extraction.md`; C9 ignored-log scan: PacketReader 86 groupchat/86 conference hits; Smark 24 groupchat/658 conference/631 muc hits | C28 adds the Go sidecar client wrapper; the sidecar still needs a bounded display-entity operation before DB-backed group/member names can be exposed | C29 bounded group display-entity extraction | | `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas via `internal/isphere.ListFilesFromMessages` and `isphere_receive_files` list mode; configured PacketReader file or directory source; `zyl\importal\` cache entries remain unlinked | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation; future UIA/client connector for download | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#c12-file-transfer-evidence-precheck` | C22 verifies safe file-list contract args; no log-to-cache/download mapping yet | defer download mapping; precheck contact/group display-name evidence | | `isphere_send_message` | none validated yet | existing bridge/API/local service preferred; UIA write connector only after internal-sandbox action evidence; network/protocol connector only after protocol discovery | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | C15 blocked: contract exists, but no committed bridge/API/local service, helper write op, UIA action chain, or protocol connector is validated | do not implement write behavior; harden existing read source configuration | | `isphere_send_file` | none validated yet | depends on send-message connector plus outbound file/upload evidence | `docs/source-discovery/2026-07-10-send-message-source-precheck.md` | blocked behind `isphere_send_message` connector selection and file upload/source policy | defer until send-message connector is validated | @@ -39,4 +39,4 @@ Start Stage C with a narrow log-backed `isphere_receive_messages` source abstrac ## Deferred source work -`MsgLib.db` has a validated copied read-only open path through the bundled 32-bit `System.Data.SQLite.dll` and password `123`. C27 adds `MsgLibReadSidecar` as the bounded x86 .NET reader boundary; the next step is a Go sidecar client wrapper and then contact/group display-name integration. +`MsgLib.db` has a validated copied read-only open path through the bundled 32-bit `System.Data.SQLite.dll` and password `123`. C27 adds `MsgLibReadSidecar` as the bounded x86 .NET reader boundary, and C28 adds the Go `internal/msglib` process client. The next step is a bounded display-entity operation before contact/group display-name integration. diff --git a/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md b/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md index f7b99b4..794e4e7 100644 --- a/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md +++ b/docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md @@ -2174,10 +2174,11 @@ Bounded MsgLib sidecar result: **Goal:** Add a Go-side client wrapper that can call the C27 x86 sidecar through a bounded JSON process contract, without exposing a new MCP tool or reading message bodies yet. -**Planned files:** +**Files:** - Create: `internal/msglib/sidecar_client.go` - Create: `internal/msglib/sidecar_client_test.go` -- Optionally update `docs/msglib-read-sidecar-contract.md` +- Modify: `docs/msglib-read-sidecar-contract.md` +- Modify: `docs/source-discovery/capability-source-matrix.md` - Modify: `docs/current-status-card.md` - Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md` @@ -2191,25 +2192,89 @@ Bounded MsgLib sidecar result: - Expose only Go functions for `SelfCheck` and `DisplaySources` metadata. - Do not add message-body reads, file downloads, sends, writes, hooks, injection, DB mutation, or a public MCP tool in this loop. -- [ ] **Step 1: Write failing fake-sidecar client tests** +- [x] **Step 1: Write failing fake-sidecar client tests** -Test request envelope, timeout/error handling, and display-source metadata decoding against a fake process. +Created `internal/msglib/sidecar_client_test.go` with fake process tests for request envelope, `self_check`, `display_sources`, structured sidecar errors, timeout handling, and env config. Verified RED with undefined `Config`, `Client`, `Protocol`, and `SidecarError` symbols. -- [ ] **Step 2: Implement sidecar client** +- [x] **Step 2: Implement sidecar client** -Use `context.Context`, `exec.CommandContext`, stdin JSON, stdout JSON, stderr capture, and small typed structs. +Created `internal/msglib/sidecar_client.go` with protocol constants, `ConfigFromEnv`, `NewClient`, `SelfCheck`, `DisplaySources`, request ids, `exec.CommandContext`, stdin/stdout JSON, timeout, stderr capture, response validation, and typed `SidecarError` handling. -- [ ] **Step 3: Verify and update docs** +- [x] **Step 3: Verify and update docs** -Run Go tests and update status/plan. Keep real evidence-backed sidecar smoke optional through `scripts/verify-msglib-sidecar.ps1`. +Ran package tests for `internal/msglib` and updated the sidecar contract, capability matrix, current status card, and this plan. Real evidence-backed sidecar smoke remains covered by `scripts\verify-msglib-sidecar.ps1` rather than normal Go unit tests. -- [ ] **Step 4: Full verification and commit** +- [x] **Step 4: Full verification and commit** + +Run before commit: + +```powershell +git diff --check +go test ./... +go build ./cmd/isphere-mcp +powershell -NoProfile -ExecutionPolicy Bypass -File scripts\build-msglib-sidecar.ps1 +powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1 +powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1 # with copied DB env paths +powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-go-mcp.ps1 +``` + +Final verification passed: `git diff --check`, `go test ./...`, explicit `go build ./cmd/isphere-mcp`, default sidecar smoke, evidence-backed sidecar smoke, and `scripts\verify-go-mcp.ps1`. + +--- + +## Loop C28 Result + +Go sidecar client result: + +- Added `internal/msglib` as the internal Go process client for `isphere.msglib.v1`. +- Added fake-sidecar unit coverage for envelope validation, `SelfCheck`, `DisplaySources`, structured `SidecarError`, timeout behavior, and env config. +- `ConfigFromEnv` reads `ISPHERE_MSGLIB_SIDECAR_EXE`, `ISPHERE_MSGLIB_SQLITE_DLL`, `ISPHERE_MSGLIB_DB`, and optional `ISPHERE_MSGLIB_PASSWORD`. +- C28 still does not register a new MCP tool, does not read message bodies, does not return contact/group row values, and does not implement file download/send/write behavior. +- Next gap: C29 must add a bounded display-entity operation before DB-backed contact/group display names can be exposed. + +--- + +## Loop C29: Bounded `MsgLib.db` display entity extraction + +**Goal:** Extend the sidecar/client boundary from schema/display-source availability to bounded contact/group display-entity metadata, so a later loop can enrich `isphere_search_contacts` and `isphere_search_groups` without returning raw rows, message bodies, file paths, downloads, sends, writes, hooks, injection, or DB mutations. + +**Planned files:** +- Modify: `native/MsgLibReadSidecar/Program.cs` +- Modify: `docs/msglib-read-sidecar-contract.md` +- Modify: `internal/msglib/sidecar_client.go` +- Modify: `internal/msglib/sidecar_client_test.go` +- Modify: `scripts/verify-msglib-sidecar.ps1` +- Modify: `docs/current-status-card.md` +- Modify: `docs/source-discovery/capability-source-matrix.md` +- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md` + +**Planned behavior:** +- Add a sidecar operation tentatively named `display_entities`. +- Request args: `sqlite_dll_path`, `db_path`, optional `password`, `entity_type` (`contacts` or `groups`), optional `query`, and bounded `limit`. +- Response data: normalized metadata only, such as `entity_type`, `source_table`, `jid`, `display_name`, `confidence`, and `matched_columns`. +- Never return message bodies, file paths, raw SQL row maps, attachment contents, send/write capabilities, or unbounded result sets. +- Use tests first. Normal Go tests should still use fake sidecar data; real copied DB verification stays opt-in through script/env paths. + +- [ ] **Step 1: Write failing display-entity contract tests** + +Add sidecar/client tests for bounded contact/group entity decoding, query/limit validation, and rejection of unsupported entity types. + +- [ ] **Step 2: Implement sidecar `display_entities` operation** + +Implement allowlisted table/column reads only for contact/group identity metadata, with parameterized query/limit and no raw row dumps. + +- [ ] **Step 3: Extend Go client and verification script** + +Add Go wrapper types/methods and optional evidence-backed verification that reports only counts/source-table names, not personal values. + +- [ ] **Step 4: Update docs, run full verification, and commit** Run: ```powershell git diff --check go test ./... +go build ./cmd/isphere-mcp powershell -NoProfile -ExecutionPolicy Bypass -File scripts\build-msglib-sidecar.ps1 powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-msglib-sidecar.ps1 powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-go-mcp.ps1 diff --git a/internal/msglib/sidecar_client.go b/internal/msglib/sidecar_client.go new file mode 100644 index 0000000..db7ab82 --- /dev/null +++ b/internal/msglib/sidecar_client.go @@ -0,0 +1,231 @@ +package msglib + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "os" + "os/exec" + "strings" + "sync/atomic" + "time" +) + +const ( + Protocol = "isphere.msglib.v1" + DefaultPassword = "123" +) + +var requestCounter uint64 + +// Config describes the local sidecar process and copied MsgLib DB inputs. +type Config struct { + SidecarPath string + SidecarArgs []string + SQLiteDLLPath string + DBPath string + Password string + Timeout time.Duration + ExtraEnv []string +} + +// ConfigFromEnv reads the C28 environment shape without wiring it into MCP tools. +func ConfigFromEnv() (Config, error) { + cfg := Config{ + SidecarPath: strings.TrimSpace(os.Getenv("ISPHERE_MSGLIB_SIDECAR_EXE")), + SQLiteDLLPath: strings.TrimSpace(os.Getenv("ISPHERE_MSGLIB_SQLITE_DLL")), + DBPath: strings.TrimSpace(os.Getenv("ISPHERE_MSGLIB_DB")), + Password: strings.TrimSpace(os.Getenv("ISPHERE_MSGLIB_PASSWORD")), + } + if cfg.Password == "" { + cfg.Password = DefaultPassword + } + if cfg.SidecarPath == "" { + return cfg, errors.New("ISPHERE_MSGLIB_SIDECAR_EXE is not configured") + } + return cfg, nil +} + +// NewClient returns a process-backed MsgLib sidecar client. +func NewClient(cfg Config) *Client { + if cfg.Password == "" { + cfg.Password = DefaultPassword + } + if cfg.Timeout == 0 { + cfg.Timeout = 10 * time.Second + } + return &Client{config: cfg} +} + +type Client struct { + config Config +} + +type SelfCheckResult struct { + SidecarName string `json:"sidecar_name"` + SidecarVersion string `json:"sidecar_version"` + Protocol string `json:"protocol"` + Runtime string `json:"runtime"` + ProcessBits int `json:"process_bits"` + ReadOnly bool `json:"read_only"` + MutatesDB bool `json:"mutates_db"` +} + +type DisplaySource struct { + Source string `json:"source"` + Table string `json:"table"` + Available bool `json:"available"` + RequiredColumns []string `json:"required_columns"` + PresentColumns []string `json:"present_columns"` +} + +type SidecarError struct { + Code string `json:"code"` + Message string `json:"message"` +} + +func (e *SidecarError) Error() string { + if e == nil { + return "msglib sidecar error" + } + if e.Code == "" { + return "msglib sidecar error: " + e.Message + } + if e.Message == "" { + return "msglib sidecar error: " + e.Code + } + return fmt.Sprintf("msglib sidecar error %s: %s", e.Code, e.Message) +} + +func (c *Client) SelfCheck(ctx context.Context) (SelfCheckResult, error) { + var out SelfCheckResult + err := c.call(ctx, "self_check", map[string]any{}, &out) + return out, err +} + +func (c *Client) DisplaySources(ctx context.Context) ([]DisplaySource, error) { + args := map[string]any{ + "sqlite_dll_path": c.config.SQLiteDLLPath, + "db_path": c.config.DBPath, + "password": c.config.Password, + } + var out displaySourcesData + if err := c.call(ctx, "display_sources", args, &out); err != nil { + return nil, err + } + return out.DisplaySources, nil +} + +type displaySourcesData struct { + DisplaySources []DisplaySource `json:"display_sources"` +} + +type requestEnvelope struct { + Protocol string `json:"protocol"` + RequestID string `json:"request_id"` + Op string `json:"op"` + Args map[string]any `json:"args,omitempty"` +} + +type responseEnvelope struct { + Protocol string `json:"protocol"` + RequestID string `json:"request_id"` + Op string `json:"op"` + OK bool `json:"ok"` + Data json.RawMessage `json:"data"` + Error *SidecarError `json:"error"` + Warnings []string `json:"warnings"` +} + +func (c *Client) call(ctx context.Context, op string, args map[string]any, out any) error { + if ctx == nil { + ctx = context.Background() + } + if strings.TrimSpace(c.config.SidecarPath) == "" { + return errors.New("msglib sidecar path is not configured") + } + + requestID := nextRequestID() + req := requestEnvelope{ + Protocol: Protocol, + RequestID: requestID, + Op: op, + Args: args, + } + payload, err := json.Marshal(req) + if err != nil { + return fmt.Errorf("marshal msglib sidecar request: %w", err) + } + payload = append(payload, '\n') + + callCtx := ctx + var cancel context.CancelFunc + if c.config.Timeout > 0 { + callCtx, cancel = context.WithTimeout(ctx, c.config.Timeout) + defer cancel() + } + + cmd := exec.CommandContext(callCtx, c.config.SidecarPath, c.config.SidecarArgs...) + cmd.Stdin = bytes.NewReader(payload) + var stdout bytes.Buffer + var stderr bytes.Buffer + cmd.Stdout = &stdout + cmd.Stderr = &stderr + if len(c.config.ExtraEnv) > 0 { + cmd.Env = append(os.Environ(), c.config.ExtraEnv...) + } + + runErr := cmd.Run() + if callCtx.Err() != nil { + if errors.Is(callCtx.Err(), context.DeadlineExceeded) { + return fmt.Errorf("msglib sidecar %s timed out: %w", op, context.DeadlineExceeded) + } + return fmt.Errorf("msglib sidecar %s context failed: %w", op, callCtx.Err()) + } + if runErr != nil { + stderrText := strings.TrimSpace(stderr.String()) + if stderrText == "" { + return fmt.Errorf("msglib sidecar %s process failed: %w", op, runErr) + } + return fmt.Errorf("msglib sidecar %s process failed: %w: %s", op, runErr, stderrText) + } + + body := bytes.TrimSpace(stdout.Bytes()) + if len(body) == 0 { + return fmt.Errorf("msglib sidecar %s returned empty stdout", op) + } + + var resp responseEnvelope + if err := json.Unmarshal(body, &resp); err != nil { + return fmt.Errorf("decode msglib sidecar %s response: %w", op, err) + } + if resp.Protocol != Protocol { + return fmt.Errorf("msglib sidecar %s response protocol = %q, want %q", op, resp.Protocol, Protocol) + } + if resp.RequestID != requestID { + return fmt.Errorf("msglib sidecar %s response request_id = %q, want %q", op, resp.RequestID, requestID) + } + if resp.Op != op { + return fmt.Errorf("msglib sidecar response op = %q, want %q", resp.Op, op) + } + if !resp.OK { + if resp.Error != nil { + return fmt.Errorf("msglib sidecar %s failed: %w", op, resp.Error) + } + return fmt.Errorf("msglib sidecar %s failed", op) + } + if out == nil || len(resp.Data) == 0 || bytes.Equal(resp.Data, []byte("null")) { + return nil + } + if err := json.Unmarshal(resp.Data, out); err != nil { + return fmt.Errorf("decode msglib sidecar %s data: %w", op, err) + } + return nil +} + +func nextRequestID() string { + n := atomic.AddUint64(&requestCounter, 1) + return fmt.Sprintf("go-%d-%d", time.Now().UnixNano(), n) +} diff --git a/internal/msglib/sidecar_client_test.go b/internal/msglib/sidecar_client_test.go new file mode 100644 index 0000000..4eb2463 --- /dev/null +++ b/internal/msglib/sidecar_client_test.go @@ -0,0 +1,217 @@ +package msglib + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os" + "strings" + "testing" + "time" +) + +func TestSelfCheckSendsProtocolEnvelopeAndDecodesResponse(t *testing.T) { + client := newFakeClient(t, "self_check_ok", Config{}) + + got, err := client.SelfCheck(context.Background()) + if err != nil { + t.Fatalf("SelfCheck returned error: %v", err) + } + if got.SidecarName != "MsgLibReadSidecar" { + t.Fatalf("SidecarName = %q, want MsgLibReadSidecar", got.SidecarName) + } + if got.Protocol != Protocol { + t.Fatalf("Protocol = %q, want %q", got.Protocol, Protocol) + } + if got.ProcessBits != 32 || !got.ReadOnly || got.MutatesDB { + t.Fatalf("unexpected safety fields: bits=%d read_only=%v mutates_db=%v", got.ProcessBits, got.ReadOnly, got.MutatesDB) + } +} + +func TestDisplaySourcesPassesConfiguredPathsAndDecodesMetadata(t *testing.T) { + cfg := Config{ + SQLiteDLLPath: `C:\fixture\System.Data.SQLite.dll`, + DBPath: `C:\fixture\MsgLib.db`, + Password: "123", + } + client := newFakeClient(t, "display_sources_ok", cfg) + + got, err := client.DisplaySources(context.Background()) + if err != nil { + t.Fatalf("DisplaySources returned error: %v", err) + } + if len(got) != 2 { + t.Fatalf("len(DisplaySources) = %d, want 2", len(got)) + } + if got[0].Source != "contacts_roster" || got[0].Table != "TD_Roster" || !got[0].Available { + t.Fatalf("unexpected first display source: %+v", got[0]) + } + if strings.Join(got[0].RequiredColumns, ",") != "UserJID,Name" { + t.Fatalf("required columns = %#v", got[0].RequiredColumns) + } + if got[1].Source != "work_group_auth" || got[1].Available { + t.Fatalf("unexpected second display source: %+v", got[1]) + } +} + +func TestClientReturnsStructuredSidecarError(t *testing.T) { + client := newFakeClient(t, "sidecar_error", Config{}) + + _, err := client.SelfCheck(context.Background()) + if err == nil { + t.Fatal("SelfCheck returned nil error, want sidecar error") + } + var sidecarErr *SidecarError + if !errors.As(err, &sidecarErr) { + t.Fatalf("error %T %[1]v does not wrap *SidecarError", err) + } + if sidecarErr.Code != "DB_OPEN_FAILED" || !strings.Contains(sidecarErr.Message, "read-only open failed") { + t.Fatalf("unexpected sidecar error: %+v", sidecarErr) + } +} + +func TestClientTimeoutStopsSidecar(t *testing.T) { + client := newFakeClient(t, "hang", Config{Timeout: 150 * time.Millisecond}) + + _, err := client.SelfCheck(context.Background()) + if err == nil { + t.Fatal("SelfCheck returned nil error, want timeout") + } + if !errors.Is(err, context.DeadlineExceeded) && !strings.Contains(strings.ToLower(err.Error()), "deadline") { + t.Fatalf("SelfCheck error = %v, want deadline exceeded", err) + } +} + +func TestConfigFromEnvReadsSidecarShape(t *testing.T) { + t.Setenv("ISPHERE_MSGLIB_SIDECAR_EXE", `C:\tools\MsgLibReadSidecar.exe`) + t.Setenv("ISPHERE_MSGLIB_SQLITE_DLL", `C:\tools\System.Data.SQLite.dll`) + t.Setenv("ISPHERE_MSGLIB_DB", `C:\copy\MsgLib.db`) + + cfg, err := ConfigFromEnv() + if err != nil { + t.Fatalf("ConfigFromEnv returned error: %v", err) + } + if cfg.SidecarPath != `C:\tools\MsgLibReadSidecar.exe` { + t.Fatalf("SidecarPath = %q", cfg.SidecarPath) + } + if cfg.SQLiteDLLPath != `C:\tools\System.Data.SQLite.dll` || cfg.DBPath != `C:\copy\MsgLib.db` { + t.Fatalf("unexpected DB config: %+v", cfg) + } + if cfg.Password != "123" { + t.Fatalf("Password = %q, want default 123", cfg.Password) + } +} + +func newFakeClient(t *testing.T, mode string, overrides Config) *Client { + t.Helper() + exe, err := os.Executable() + if err != nil { + t.Fatalf("os.Executable: %v", err) + } + cfg := overrides + cfg.SidecarPath = exe + cfg.SidecarArgs = []string{"-test.run=TestFakeSidecarProcess", "--"} + cfg.ExtraEnv = append(cfg.ExtraEnv, "ISPHERE_MSGLIB_FAKE_SIDECAR=1", "ISPHERE_MSGLIB_FAKE_MODE="+mode) + if cfg.Timeout == 0 { + cfg.Timeout = 2 * time.Second + } + return NewClient(cfg) +} + +func TestFakeSidecarProcess(t *testing.T) { + if os.Getenv("ISPHERE_MSGLIB_FAKE_SIDECAR") != "1" { + return + } + os.Exit(fakeSidecarMain()) +} + +func fakeSidecarMain() int { + mode := os.Getenv("ISPHERE_MSGLIB_FAKE_MODE") + if mode == "hang" { + time.Sleep(5 * time.Second) + return 0 + } + + var req map[string]any + if err := json.NewDecoder(os.Stdin).Decode(&req); err != nil { + fmt.Fprintf(os.Stderr, "decode request: %v\n", err) + return 2 + } + protocol, _ := req["protocol"].(string) + op, _ := req["op"].(string) + requestID, _ := req["request_id"].(string) + if protocol != Protocol || op == "" || requestID == "" { + fmt.Fprintf(os.Stderr, "bad envelope: protocol=%q op=%q request_id=%q\n", protocol, op, requestID) + return 2 + } + + if mode == "sidecar_error" { + return writeFakeResponse(map[string]any{ + "protocol": Protocol, + "request_id": requestID, + "op": op, + "ok": false, + "data": nil, + "error": map[string]any{ + "code": "DB_OPEN_FAILED", + "message": "read-only open failed", + }, + "warnings": []any{}, + }) + } + + switch op { + case "self_check": + return writeFakeResponse(map[string]any{ + "protocol": Protocol, + "request_id": requestID, + "op": op, + "ok": true, + "data": map[string]any{ + "sidecar_name": "MsgLibReadSidecar", + "sidecar_version": "0.1.0", + "protocol": Protocol, + "process_bits": 32, + "read_only": true, + "mutates_db": false, + }, + "error": nil, + "warnings": []any{}, + }) + case "display_sources": + args, _ := req["args"].(map[string]any) + if args["sqlite_dll_path"] != `C:\fixture\System.Data.SQLite.dll` || args["db_path"] != `C:\fixture\MsgLib.db` || args["password"] != "123" { + fmt.Fprintf(os.Stderr, "bad display_sources args: %#v\n", args) + return 2 + } + return writeFakeResponse(map[string]any{ + "protocol": Protocol, + "request_id": requestID, + "op": op, + "ok": true, + "data": map[string]any{ + "metadata_only": true, + "read_only": true, + "message_body_values_returned": false, + "display_sources": []any{ + map[string]any{"source": "contacts_roster", "table": "TD_Roster", "available": true, "required_columns": []any{"UserJID", "Name"}, "present_columns": []any{"UserJID", "Name", "Mobile"}}, + map[string]any{"source": "work_group_auth", "table": "TD_WorkGroupAuth", "available": false, "required_columns": []any{"GroupJID", "GroupName"}, "present_columns": []any{"GroupJID"}}, + }, + }, + "error": nil, + "warnings": []any{}, + }) + default: + fmt.Fprintf(os.Stderr, "unexpected op: %s\n", op) + return 2 + } +} + +func writeFakeResponse(resp map[string]any) int { + if err := json.NewEncoder(os.Stdout).Encode(resp); err != nil { + fmt.Fprintf(os.Stderr, "encode response: %v\n", err) + return 2 + } + return 0 +}