feat: load packet logs from configured source
This commit is contained in:
zhaoyilun
@@ -21,13 +21,13 @@ Remote base before local roadmap commits: `b2d839e Merge branch 'codex/n15-repor
|
||||
- `win_helper_dump_uia`
|
||||
- `isphere_receive_messages`
|
||||
- C# `ISphereWinHelper` exists under `native/ISphereWinHelper` and uses the `isphere.helper.v1` stdin/stdout JSON contract.
|
||||
- `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, five-tool MCP surface, and empty-source `isphere_receive_messages` smoke call.
|
||||
- `scripts/verify-win-helper.ps1` and `scripts/verify-go-mcp.ps1` provide repeatable local checks for the helper, five-tool MCP surface, and default empty-source `isphere_receive_messages` smoke call.
|
||||
- N12-pre and N12R documents define safe offline evidence intake and internal-sandbox live UIA capture procedures.
|
||||
- N13/N14/N15 produced selector catalog/report validation infrastructure, but those are supporting validation assets only.
|
||||
|
||||
## What cannot be claimed externally
|
||||
|
||||
- The project has not yet implemented business MCP tools for contact search, group search, message sending, file receiving, or file sending; `isphere_receive_messages` is registered but still lacks a production loader/configuration path.
|
||||
- The project has not yet implemented business MCP tools for contact search, group search, message sending, file receiving, or file sending; `isphere_receive_messages` has an operator-local encrypted log-file loader but not a production ingestion path.
|
||||
- The project has not completed production iSphere integration.
|
||||
- The project has not proven a stable real message source for the full business workflow beyond decrypted PacketReader log evidence and synthetic/redacted tests.
|
||||
- The project has not implemented real message sending or real file upload.
|
||||
@@ -40,7 +40,7 @@ Remote base before local roadmap commits: `b2d839e Merge branch 'codex/n15-repor
|
||||
- The helper protocol boundary is explicit and keeps Windows/UIA work separate from the Go MCP service layer.
|
||||
- The current code can support future MCP business tools by routing low-level Windows/UIA fallback work through a bounded helper layer.
|
||||
- The project has repeatable verification scripts for the helper/MCP foundation and the current five-tool surface.
|
||||
- The project now has a corrected business contract and plan for core MCP communication tools, plus a registered first read tool for receiving messages from a future configured source.
|
||||
- The project now has a corrected business contract and plan for core MCP communication tools, plus a registered first read tool that can use `ISPHERE_PACKET_LOG_FILE` for operator-local encrypted PacketReader log lines.
|
||||
|
||||
## N13/N14/N15 positioning
|
||||
|
||||
@@ -48,7 +48,7 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme
|
||||
|
||||
## Current loop
|
||||
|
||||
Current loop: `Stage C / Loop C4 - operator verification and docs`.
|
||||
Current loop: `Stage C / Loop C6 - fixture-backed operator smoke for configured receive source`.
|
||||
|
||||
Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`.
|
||||
|
||||
@@ -64,12 +64,14 @@ Current loop output so far:
|
||||
1. C1: `internal/isphere/logcodec.DecryptLine` and `internal/isphere/packetlog.ParseMessageLog` for redacted PacketReader message logs.
|
||||
2. C2: `internal/isphere.EncryptedPacketLogSource.ReceiveMessages` source abstraction with synthetic/redacted encrypted fixtures.
|
||||
3. C3: `isphere_receive_messages` MCP tool registration and server tool list update.
|
||||
4. C4: updating repeatable operator verification and docs for the five-tool surface.
|
||||
4. C4: repeatable operator verification and docs for the five-tool surface.
|
||||
5. C5: safe `ISPHERE_PACKET_LOG_FILE` loader for operator-local encrypted PacketReader log lines.
|
||||
6. C6: fixture-backed operator smoke for configured receive source.
|
||||
|
||||
## Next business mainline
|
||||
|
||||
1. Finish Stage C Loop C4: verify and document the five-tool MCP surface.
|
||||
2. Implement the next loop: safe configurable source loader or fixture path for `isphere_receive_messages`, without committing raw evidence.
|
||||
1. Finish Stage C Loop C6: add fixture-backed configured-source operator smoke.
|
||||
2. Rewrite the next loop around either production-safe source configuration hardening or the next read capability, based on C6 results.
|
||||
3. Add contact/group/file read tools after their source decisions are recorded.
|
||||
4. Add `isphere_send_message` and `isphere_send_file` after source discovery and connector selection.
|
||||
5. Keep UIA selector/report work as fallback support, not as the primary roadmap.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
This runbook is for the first Go MCP phase of `isphere-ai-bridge`. It lets an operator build and verify the Windows helper and the Go MCP server without reading the source code.
|
||||
|
||||
Current scope: expose four WinHelper observation operations plus the first log-backed business read tool, `isphere_receive_messages`, through Go MCP. The receive tool is registered with an empty default source until a safe evidence loader is wired.
|
||||
Current scope: expose four WinHelper observation operations plus the first log-backed business read tool, `isphere_receive_messages`, through Go MCP. The receive tool is registered with an empty default source unless `ISPHERE_PACKET_LOG_FILE` points to an operator-local encrypted PacketReader log-line file.
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
@@ -96,7 +96,25 @@ The verification confirms:
|
||||
- Verification uses synthetic/local helper checks and does not load raw N12-pre evidence.
|
||||
- Contact/group/file/search/send business tools remain later-stage work.
|
||||
|
||||
## 6. Configure MCP client command
|
||||
## 6. Configure optional message source
|
||||
|
||||
By default, `isphere_receive_messages` is registered but uses an empty source, so it returns:
|
||||
|
||||
```json
|
||||
{"messages":[]}
|
||||
```
|
||||
|
||||
To point the server at a local encrypted PacketReader log-line file, set:
|
||||
|
||||
```powershell
|
||||
$env:ISPHERE_PACKET_LOG_FILE = "E:\coding\codex\isphere-ai-bridge\runs\offline-evidence-intake\<evidence-id>\packet-reader-lines.txt"
|
||||
```
|
||||
|
||||
The file must contain one encrypted Base64 log line per line. Blank lines are ignored. Keep this file under ignored `runs/` paths or another operator-local location; do not commit raw logs or decrypted message content.
|
||||
|
||||
Then start the MCP server normally. The command entry point uses `ISPHERE_PACKET_LOG_FILE` if it is set; the repeatable verification script clears it so the default smoke test remains deterministic.
|
||||
|
||||
## 7. Configure MCP client command
|
||||
|
||||
After building a stable binary, configure your MCP client to run the Go MCP executable as a stdio server.
|
||||
|
||||
@@ -131,7 +149,7 @@ If you used `go build ./cmd/isphere-mcp` without `-o`, point the command to:
|
||||
E:\coding\codex\isphere-ai-bridge\isphere-mcp.exe
|
||||
```
|
||||
|
||||
## 7. Allowed tools
|
||||
## 8. Allowed tools
|
||||
|
||||
These five tools are currently allowed:
|
||||
|
||||
@@ -151,7 +169,7 @@ Allowed parameters:
|
||||
- `win_helper_dump_uia`: `hwnd`, `max_depth`, `include_text`, `max_children` only.
|
||||
- `isphere_receive_messages`: `limit` only.
|
||||
|
||||
## 8. Current tool surface and next-stage route
|
||||
## 9. Current tool surface and next-stage route
|
||||
|
||||
The current runbook verifies five tools:
|
||||
|
||||
@@ -161,11 +179,11 @@ The current runbook verifies five tools:
|
||||
- `win_helper_dump_uia`
|
||||
- `isphere_receive_messages`
|
||||
|
||||
`isphere_receive_messages` currently has parser/source/tool registration and an empty default server source. Raw N12-pre evidence remains under ignored `runs/` paths and is not committed. The next implementation slice should wire a safe configurable loader or fixture path before claiming production message ingestion.
|
||||
`isphere_receive_messages` currently has parser/source/tool registration and an empty default server source. Raw N12-pre evidence remains under ignored `runs/` paths and is not committed. The current command entry point can read an operator-local encrypted PacketReader log-line file via `ISPHERE_PACKET_LOG_FILE`; this is still not a production ingestion claim because raw evidence remains local and uncommitted.
|
||||
|
||||
Core business tools for contacts, groups, messages, and files are defined in `docs/mcp-core-tools-contract.md`. Send/file capabilities enter the roadmap through that contract after source discovery, connector selection, preview metadata, execution metadata, and audit records are in place.
|
||||
|
||||
## 9. Real-login UIA capture procedure
|
||||
## 10. Real-login UIA capture procedure
|
||||
|
||||
This is a later real-login capture gate used to collect live UI structure for connector work.
|
||||
|
||||
@@ -206,7 +224,7 @@ Procedure:
|
||||
|
||||
8. Review and redact any sensitive content before sharing reports.
|
||||
|
||||
## 10. If this outer-network environment cannot log in
|
||||
## 11. If this outer-network environment cannot log in
|
||||
|
||||
iSphere is internal-network only. If the current repository environment is outside that network, use two separate routes:
|
||||
|
||||
@@ -237,7 +255,7 @@ powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\validate-n12r-retu
|
||||
|
||||
Keep the validation report outside `$captureRoot`; the returned package is evidence input and should remain unchanged.
|
||||
|
||||
## 11. Troubleshooting
|
||||
## 12. Troubleshooting
|
||||
|
||||
- If C# helper build fails, run `scripts\build-win-helper.ps1` directly and check for missing .NET Framework reference assemblies.
|
||||
- If `win_helper_version` fails, rerun `powershell -NoProfile -ExecutionPolicy Bypass -File scripts\verify-win-helper.ps1` first.
|
||||
|
||||
@@ -17,7 +17,7 @@ Schema notes: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`
|
||||
|
||||
| MCP tool | Primary source | Fallback source | Evidence file | Decision status | Next implementation slice |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; wrapped `MsgLib.db` after DB wrapper is solved | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence` | C3 registered and unit-verified; default server source is empty until operator verification/loader wiring | fixture-backed operator verification and docs |
|
||||
| `isphere_receive_messages` | decrypted `PacketReader.ProcessPacket` XMPP `<message>` stanzas via `internal/isphere.EncryptedPacketLogSource` | decrypted `Smark.SendReceive` transport stanzas; decrypted `SaveToDB` `Chat_OnMessageArrived` traces; wrapped `MsgLib.db` after DB wrapper is solved | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence` | C5 env loader wired through ISPHERE_PACKET_LOG_FILE; default source remains empty when unset | fixture-backed configured-source operator smoke |
|
||||
| `isphere_search_contacts` | decrypted packet/send-receive JIDs and later roster stanzas from `Smark.SendReceive` | `MsgLib.db` contact/conversation tables after DB wrapper is solved; UIA helper source if display names are missing | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#log-decryption-result` | candidate pending parser coverage | second |
|
||||
| `isphere_search_groups` | decrypted XMPP group/message stanzas if groupchat/group JIDs appear | `MsgLib.db` group/conversation tables after DB wrapper is solved; UIA helper source if group display names are missing | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence` | candidate pending group stanza evidence | third |
|
||||
| `isphere_receive_files` | decrypted `PacketReader.ProcessPacket` file-transfer message stanzas plus `zyl\importal\<hash>` cache mapping | decrypted `Smark.SendReceive` and `SaveToDB` traces for file-reference reconciliation | `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md#field-mapping-evidence` | candidate pending cache-reference mapping | fourth |
|
||||
|
||||
@@ -795,7 +795,7 @@ Implemented and verified in this loop:
|
||||
- Configured behavior: env var set to a local text file means the server reads non-empty encrypted Base64 lines from that file into `isphere.EncryptedPacketLogSource{Lines: lines}`.
|
||||
- Evidence rule: the configured file should live under ignored `runs/` or another operator-local path; the loader must not require or commit raw evidence.
|
||||
|
||||
- [ ] **Step 1: Write failing file loader test**
|
||||
- [x] **Step 1: Write failing file loader test**
|
||||
|
||||
Create `internal/isphere/file_source_test.go` first and run:
|
||||
|
||||
@@ -805,23 +805,23 @@ go test ./internal/isphere -run TestLoadEncryptedPacketLogSourceFromFileReadsNon
|
||||
|
||||
Expected initial failure: loader function/type does not exist.
|
||||
|
||||
- [ ] **Step 2: Implement file loader**
|
||||
- [x] **Step 2: Implement file loader**
|
||||
|
||||
Implement a small loader that reads a UTF-8 text file, trims whitespace, skips blank lines, and returns `EncryptedPacketLogSource{Lines: lines}`. It should return clear path/context errors for missing or unreadable files.
|
||||
|
||||
- [ ] **Step 3: Write failing env-wired server test**
|
||||
- [x] **Step 3: Write failing env-wired server test**
|
||||
|
||||
Update `internal/mcpserver/server_test.go` with a temp encrypted fixture file and a test that sets `ISPHERE_PACKET_LOG_FILE`, constructs the env-wired server, calls `isphere_receive_messages`, and expects one redacted message. Run the specific test before implementation.
|
||||
|
||||
- [ ] **Step 4: Implement env-wired server construction**
|
||||
- [x] **Step 4: Implement env-wired server construction**
|
||||
|
||||
Add `NewServerFromEnv()` or equivalent env-aware construction. Keep `NewServer()` as the empty default for existing tests. Update `cmd/isphere-mcp/main.go` to use the env-aware server.
|
||||
|
||||
- [ ] **Step 5: Update operator verification/docs**
|
||||
- [x] **Step 5: Update operator verification/docs**
|
||||
|
||||
Extend `scripts/verify-go-mcp.ps1` so default verification still asserts zero messages when no env var is set. Document optional `ISPHERE_PACKET_LOG_FILE` usage in `docs/go-mcp-runbook.md` and update the status card.
|
||||
|
||||
- [ ] **Step 6: Run full verification and commit**
|
||||
- [x] **Step 6: Run full verification and commit**
|
||||
|
||||
Run:
|
||||
|
||||
@@ -836,6 +836,67 @@ Expected: all exit 0. Remove any generated root `isphere-mcp.exe` after build. C
|
||||
|
||||
---
|
||||
|
||||
## Loop C5 Result
|
||||
|
||||
Implemented and verified in this loop:
|
||||
|
||||
- `internal/isphere.LoadEncryptedPacketLogSourceFromFile(path)` reads non-empty encrypted Base64 PacketReader log lines from an operator-local text file and returns `EncryptedPacketLogSource{Lines: lines}`.
|
||||
- `internal/mcpserver.NewServerFromEnv()` reads `ISPHERE_PACKET_LOG_FILE` when set, otherwise preserves the empty default receive source.
|
||||
- `cmd/isphere-mcp/main.go` now uses the env-aware server constructor and fails fast on invalid configuration.
|
||||
- `scripts/verify-go-mcp.ps1` now exercises `NewServerFromEnv()` with `ISPHERE_PACKET_LOG_FILE` cleared and confirms the deterministic empty-source behavior.
|
||||
- Docs now explain optional `ISPHERE_PACKET_LOG_FILE` usage and reinforce that raw evidence must remain under ignored/operator-local paths.
|
||||
- Verification completed for C5 with `git diff --check`, `go test ./...`, `go build ./cmd/isphere-mcp`, and `powershell -NoProfile -ExecutionPolicy Bypass -File scripts/verify-go-mcp.ps1`; the generated root `isphere-mcp.exe` build artifact was removed.
|
||||
|
||||
---
|
||||
|
||||
## Loop C6: Fixture-backed operator smoke for configured receive source
|
||||
|
||||
**Goal:** Extend repeatable operator verification so a synthetic encrypted PacketReader fixture file proves `ISPHERE_PACKET_LOG_FILE` can drive `isphere_receive_messages` end-to-end without committing raw evidence.
|
||||
|
||||
**Planned files:**
|
||||
- Modify: `scripts/verify-go-mcp.ps1`
|
||||
- Modify: `docs/go-mcp-runbook.md`
|
||||
- Modify: `docs/current-status-card.md`
|
||||
- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`
|
||||
|
||||
**Planned behavior:**
|
||||
- Existing default smoke remains: unset `ISPHERE_PACKET_LOG_FILE` returns `messages: []`.
|
||||
- New fixture smoke creates a temporary synthetic/redacted PacketReader plaintext, encrypts it with the known log policy inside the temporary Go harness, writes one encrypted line to a temp file, sets `ISPHERE_PACKET_LOG_FILE` to that temp file, starts `mcpserver.NewServerFromEnv()`, calls `isphere_receive_messages`, and asserts one normalized message.
|
||||
- No raw N12-pre evidence or decrypted user content is written to the repo.
|
||||
|
||||
- [ ] **Step 1: Capture current fixture-smoke gap**
|
||||
|
||||
Run the current verification script and note that it only proves the default empty source, not configured fixture ingestion:
|
||||
|
||||
```powershell
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/verify-go-mcp.ps1
|
||||
```
|
||||
|
||||
Expected: PASS with `receive_message_count: 0`, but no configured-source proof yet.
|
||||
|
||||
- [ ] **Step 2: Update the temporary Go harness**
|
||||
|
||||
Extend the generated harness with a second server session that sets `ISPHERE_PACKET_LOG_FILE` to a temp synthetic encrypted log-line file and asserts one returned message with redacted content.
|
||||
|
||||
- [ ] **Step 3: Update docs**
|
||||
|
||||
Update the runbook/status card to say verification covers both default empty source and synthetic configured-source smoke.
|
||||
|
||||
- [ ] **Step 4: Run full verification and commit**
|
||||
|
||||
Run:
|
||||
|
||||
```powershell
|
||||
git diff --check
|
||||
go test ./...
|
||||
go build ./cmd/isphere-mcp
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/verify-go-mcp.ps1
|
||||
```
|
||||
|
||||
Expected: all exit 0. Remove any generated root `isphere-mcp.exe` after build. Commit exact C6 paths only and rewrite the next loop from actual results.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage: the plan follows the user's cyclic requirement: plan first, implement one loop at a time, update the next loop after each result, and ask only on blockers.
|
||||
|
||||
Reference in New Issue
Block a user