docs: plan stage c iterative receive loop

This commit is contained in:
zhaoyilun
2026-07-09 22:46:16 +08:00
parent 1eed0f62f2
commit de3bb922da
3 changed files with 499 additions and 35 deletions

View File

@@ -47,24 +47,29 @@ N13/N14/N15 are pre-business validation results. They can help identify UI eleme
## Current loop
Current loop: `Stage B1 - source discovery from N12-pre zyl package`.
Current loop: `Stage C / Loop C1 - log decryptor and PacketReader parser`.
Plan file: `docs/superpowers/plans/2026-07-09-mcp-core-source-discovery-loop.md`.
Plan file: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`.
Loop output:
Loop rule:
1. Evidence index for downloaded `zyl.rar` package.
2. Capability-source matrix for the four read tools:
- `isphere_search_contacts`
- `isphere_search_groups`
- `isphere_receive_messages`
- `isphere_receive_files`
3. Selected first Stage C implementation slice.
1. Write or update the plan before coding.
2. Implement one loop only.
3. Verify, commit exact paths, then update the next loop based on actual results.
4. Continue automatically while the next step is clear; ask the user only when blocked or when business direction changes.
Current loop output:
1. `internal/isphere/logcodec.DecryptLine` for DES/CBC/PKCS7 Base64 log lines.
2. `internal/isphere/packetlog.ParseMessageLog` for redacted `PacketReader.ProcessPacket` XMPP `<message>` stanzas.
3. Passing unit tests with synthetic/redacted fixtures only.
4. Updated Loop C2 plan before source abstraction starts.
## Next business mainline
1. Complete Stage B1 evidence index and capability-source matrix.
2. Open Stage C with `isphere_receive_messages` as the first likely slice if evidence supports it.
3. Add contact/group/file read tools after their source decisions are recorded.
4. Add `isphere_send_message` and `isphere_send_file` after source discovery and connector selection.
5. Keep UIA selector/report work as fallback support, not as the primary roadmap.
1. Complete Stage C Loop C1: log decryptor + PacketReader parser.
2. Rewrite and execute Loop C2: log-backed `ReceiveMessages` source abstraction.
3. Register `isphere_receive_messages` only after parser/source tests pass.
4. Add contact/group/file read tools after their source decisions are recorded.
5. Add `isphere_send_message` and `isphere_send_file` after source discovery and connector selection.
6. Keep UIA selector/report work as fallback support, not as the primary roadmap.

View File

@@ -1,39 +1,43 @@
# Source Discovery Ledger
Date: 2026-07-09
Current loop: Stage B1 - N12-pre zyl evidence source discovery
Current loop: Stage C / Loop C1 - log decryptor and PacketReader parser
## Objective
Find the best real capability source for the MCP core read tools before implementing Stage C.
Build the first business read capability path for `isphere_receive_messages` from decrypted `PacketReader.ProcessPacket` XMPP message logs.
## Inputs
- `runs/offline-evidence-intake/zyl-qqfile-20260709/archives/zyl.rar`
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/archive-list.txt`
- `runs/offline-evidence-intake/zyl-qqfile-20260709/metadata/sha256.txt`
- `docs/source-discovery/2026-07-09-n12-pre-zyl-index.md`
- `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`
- `docs/source-discovery/capability-source-matrix.md`
- `docs/mcp-core-tools-contract.md`
- `docs/mcp-core-business-plan.md`
## Loop cadence
## Loop protocol
1. Inventory evidence.
2. Identify source candidates.
3. Map candidates to MCP tools.
4. Select primary/fallback source per tool.
5. Open one Stage C implementation slice.
6. Verify with `go test ./...`, `go build ./cmd/isphere-mcp`, and MCP verification script.
7. Commit exact paths.
1. Write or update the plan before coding.
2. Implement one loop only.
3. Run loop-specific tests.
4. Run full verification with `go test ./...` and `go build ./cmd/isphere-mcp`.
5. Commit exact paths.
6. Update the next loop based on actual results.
7. Continue automatically while the next step is clear; ask the user only when blocked or when the business direction changes.
## Current target order
## Current loop output
1. `isphere_receive_messages`
2. `isphere_search_contacts`
3. `isphere_search_groups`
4. `isphere_receive_files`
Loop C1 must produce:
Reason: message source discovery is the current blocker; contacts/groups can be derived or selected after the message source shape is known.
1. `internal/isphere/logcodec.DecryptLine` for DES/CBC/PKCS7 Base64 log lines.
2. `internal/isphere/packetlog.ParseMessageLog` for redacted `PacketReader.ProcessPacket` XMPP `<message>` stanzas.
3. Passing unit tests with synthetic/redacted fixtures only.
4. A plan update for Loop C2 before any source abstraction code starts.
## Current plan
See `docs/superpowers/plans/2026-07-09-mcp-core-source-discovery-loop.md`.
See `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`.
## Last completed discovery result
Stage B2 showed that selected `MsgLib.db` files are not direct SQLite. Decrypted `PacketReader.ProcessPacket` XML is now the first implementable source for `isphere_receive_messages`.

View File

@@ -0,0 +1,455 @@
# Stage C Log-backed Receive Messages Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build the first business read capability for `isphere_receive_messages` using decrypted `PacketReader.ProcessPacket` XMPP message logs from the N12-pre zyl evidence package.
**Architecture:** Work in short closed loops. Each loop has one deliverable, a verification gate, a commit, and a plan-rewrite step before the next loop starts. Stage C starts with pure Go parsing code and redacted fixtures, then adds a source abstraction, then registers the MCP tool only after parser/source behavior is verified.
**Tech Stack:** Go 1.23.4, standard library `crypto/des`, `crypto/cipher`, `encoding/base64`, `encoding/xml`, existing Go MCP SDK, PowerShell verification scripts.
## Global Constraints
- Repository root: `E:\coding\codex\isphere-ai-bridge`.
- Remote name: `gitea`; active branch: `main`.
- Code search rule from `AGENTS.md`: use `git ls-files`, `ag`, or `grep -R`; do not use `rg`.
- Current evidence decision source: `docs/source-discovery/2026-07-09-n12-pre-zyl-schema-notes.md`.
- Current capability matrix: `docs/source-discovery/capability-source-matrix.md`.
- Raw evidence and extracted logs stay under `runs/`; never commit raw DB/log files or decrypted user message content.
- Tests use synthetic or redacted fixtures only.
- Every implementation loop must run `go test ./...` and `go build ./cmd/isphere-mcp` before commit.
- Every loop must update this plan or a loop notes document before the next loop starts.
- If a loop output contradicts the current plan, update the next loop rather than pushing ahead.
---
## Loop Protocol
Each loop follows the same 7-step protocol:
1. State the active loop id and one-sentence goal.
2. Read the current plan section for that loop.
3. Implement only that loop's files.
4. Run the loop-specific test first.
5. Run full verification: `go test ./...`, `go build ./cmd/isphere-mcp`.
6. Commit exact paths.
7. Rewrite the next loop section based on the actual result.
Stop and ask the user only when:
- The current loop cannot proceed because required evidence is missing.
- The actual output contradicts the product goal and no local fallback is clear.
- A decision affects business scope, such as switching from log-backed read to UIA or DB-wrapper work as the mainline.
---
## File Structure
### Loop C1 files
- Create: `E:\coding\codex\isphere-ai-bridge\internal\isphere\logcodec\codec.go`
- Create: `E:\coding\codex\isphere-ai-bridge\internal\isphere\logcodec\codec_test.go`
- Create: `E:\coding\codex\isphere-ai-bridge\internal\isphere\packetlog\message.go`
- Create: `E:\coding\codex\isphere-ai-bridge\internal\isphere\packetlog\message_test.go`
- Modify: `E:\coding\codex\isphere-ai-bridge\docs\superpowers\plans\2026-07-09-stage-c-log-backed-receive-messages-loop.md`
### Later loop files
- Create: `E:\coding\codex\isphere-ai-bridge\internal\isphere\source.go`
- Create: `E:\coding\codex\isphere-ai-bridge\internal\isphere\source_test.go`
- Create: `E:\coding\codex\isphere-ai-bridge\internal\tools\isphere_read.go`
- Create: `E:\coding\codex\isphere-ai-bridge\internal\tools\isphere_read_test.go`
- Modify: `E:\coding\codex\isphere-ai-bridge\internal\mcpserver\server.go`
- Modify: `E:\coding\codex\isphere-ai-bridge\internal\tools\winhelper_test.go`
- Modify: `E:\coding\codex\isphere-ai-bridge\docs\source-discovery\capability-source-matrix.md`
---
## Loop C1: Log decryptor and PacketReader parser
**Goal:** Produce pure Go code that decrypts one Base64 DES/CBC/PKCS7 log line and parses a redacted `PacketReader.ProcessPacket` XMPP `<message>` stanza into a normalized internal message.
**Files:**
- Create: `internal/isphere/logcodec/codec.go`
- Create: `internal/isphere/logcodec/codec_test.go`
- Create: `internal/isphere/packetlog/message.go`
- Create: `internal/isphere/packetlog/message_test.go`
- Modify: `docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md`
**Interfaces:**
- Produces: `logcodec.DecryptLine(ciphertextBase64 string) (string, error)`.
- Produces: `packetlog.ParseMessageLog(plaintext string) (packetlog.Message, error)`.
- Produces: `packetlog.Message` with fields `ID`, `From`, `To`, `Type`, `Body`, `Subject`, `SendTime`, `ReceiptID`, `ReceiptType`, `Consumed`, and `Read`.
- [ ] **Step 1: Write failing decrypt test**
Create `internal/isphere/logcodec/codec_test.go`:
```go
package logcodec
import "testing"
func TestDecryptLineWithKnownPolicyRoundTrip(t *testing.T) {
plaintext := "--------------------------------------------------------------------------------------------------------------------------------------------\r\n2026/7/7 15:30:07\r\n<message id=\"msg-1\" from=\"sender@imopenfire1-lanzhou/imp_pc_4.1.2.6842\" to=\"receiver@imopenfire1-lanzhou\" type=\"chat\"><body>redacted</body></message>"
ciphertext := encryptLineForTest(t, plaintext)
got, err := DecryptLine(ciphertext)
if err != nil {
t.Fatalf("DecryptLine returned error: %v", err)
}
if got != plaintext {
t.Fatalf("DecryptLine() = %q, want %q", got, plaintext)
}
}
```
Run:
```powershell
go test ./internal/isphere/logcodec -run TestDecryptLineWithKnownPolicyRoundTrip -v
```
Expected: FAIL because the package/function does not exist.
- [ ] **Step 2: Implement decryptor and test helper**
Create `internal/isphere/logcodec/codec.go`:
```go
package logcodec
import (
"crypto/cipher"
"crypto/des"
"encoding/base64"
"errors"
"fmt"
)
var (
policyKey = []byte("hyhccdtm")
policyIV = []byte{0x12, 0x34, 0x56, 0x78, 0x90, 0xAB, 0xCD, 0xEF}
)
func DecryptLine(ciphertextBase64 string) (string, error) {
ciphertext, err := base64.StdEncoding.DecodeString(ciphertextBase64)
if err != nil {
return "", fmt.Errorf("decode base64 log line: %w", err)
}
if len(ciphertext) == 0 || len(ciphertext)%des.BlockSize != 0 {
return "", fmt.Errorf("ciphertext length %d is not a positive DES block multiple", len(ciphertext))
}
block, err := des.NewCipher(policyKey)
if err != nil {
return "", fmt.Errorf("create DES cipher: %w", err)
}
plain := make([]byte, len(ciphertext))
cipher.NewCBCDecrypter(block, policyIV).CryptBlocks(plain, ciphertext)
plain, err = unpadPKCS7(plain, des.BlockSize)
if err != nil {
return "", err
}
return string(plain), nil
}
func unpadPKCS7(data []byte, blockSize int) ([]byte, error) {
if len(data) == 0 || len(data)%blockSize != 0 {
return nil, errors.New("pkcs7 data is empty or not block aligned")
}
pad := int(data[len(data)-1])
if pad == 0 || pad > blockSize || pad > len(data) {
return nil, fmt.Errorf("invalid pkcs7 padding length %d", pad)
}
for i := len(data) - pad; i < len(data); i++ {
if int(data[i]) != pad {
return nil, errors.New("invalid pkcs7 padding bytes")
}
}
return data[:len(data)-pad], nil
}
```
Append this helper to `internal/isphere/logcodec/codec_test.go`:
```go
func encryptLineForTest(t *testing.T, plaintext string) string {
t.Helper()
block, err := des.NewCipher(policyKey)
if err != nil {
t.Fatalf("NewCipher: %v", err)
}
plain := padPKCS7ForTest([]byte(plaintext), des.BlockSize)
ciphertext := make([]byte, len(plain))
cipher.NewCBCEncrypter(block, policyIV).CryptBlocks(ciphertext, plain)
return base64.StdEncoding.EncodeToString(ciphertext)
}
func padPKCS7ForTest(data []byte, blockSize int) []byte {
pad := blockSize - len(data)%blockSize
out := append([]byte(nil), data...)
for i := 0; i < pad; i++ {
out = append(out, byte(pad))
}
return out
}
```
Add imports in `codec_test.go`:
```go
import (
"crypto/cipher"
"crypto/des"
"encoding/base64"
"testing"
)
```
- [ ] **Step 3: Verify decryptor test passes**
Run:
```powershell
go test ./internal/isphere/logcodec -run TestDecryptLineWithKnownPolicyRoundTrip -v
```
Expected: PASS.
- [ ] **Step 4: Write failing PacketReader parser test**
Create `internal/isphere/packetlog/message_test.go`:
```go
package packetlog
import "testing"
func TestParseMessageLogExtractsXMPPMessage(t *testing.T) {
plaintext := "--------------------------------------------------------------------------------------------------------------------------------------------\r\n2026/7/7 15:30:07\r\n<message id=\"2de080bf-f5cc-45e2-b229-2df6bcd4afe3\" from=\"sender@imopenfire1-lanzhou/imp_pc_4.1.2.6842\" to=\"receiver@imopenfire1-lanzhou\" type=\"chat\">\r\n <body>redacted.docx</body>\r\n <received xmlns=\"urn:xmpp:receipts\" id=\"167c08c3d710489f812c13dc23d4e404\" type=\"1\" stamp=\"\" />\r\n <subject>FILE_TRANSFER_CANCEL</subject>\r\n <isphere xmlns=\"isphere.im\" type=\"1001\" sendtime=\"1783423807000\" version=\"1\">\r\n <alert type=\"0\" />\r\n </isphere>\r\n <consumed />\r\n <readed />\r\n</message>"
got, err := ParseMessageLog(plaintext)
if err != nil {
t.Fatalf("ParseMessageLog returned error: %v", err)
}
if got.ID != "2de080bf-f5cc-45e2-b229-2df6bcd4afe3" {
t.Fatalf("ID = %q", got.ID)
}
if got.From != "sender@imopenfire1-lanzhou/imp_pc_4.1.2.6842" || got.To != "receiver@imopenfire1-lanzhou" {
t.Fatalf("from/to = %q/%q", got.From, got.To)
}
if got.Body != "redacted.docx" || got.Subject != "FILE_TRANSFER_CANCEL" {
t.Fatalf("body/subject = %q/%q", got.Body, got.Subject)
}
if got.SendTime != "1783423807000" || got.ReceiptID != "167c08c3d710489f812c13dc23d4e404" || got.ReceiptType != "1" {
t.Fatalf("receipt fields = %+v", got)
}
if !got.Consumed || !got.Read {
t.Fatalf("consumed/read = %v/%v", got.Consumed, got.Read)
}
}
```
Run:
```powershell
go test ./internal/isphere/packetlog -run TestParseMessageLogExtractsXMPPMessage -v
```
Expected: FAIL because the package/function does not exist.
- [ ] **Step 5: Implement PacketReader parser**
Create `internal/isphere/packetlog/message.go`:
```go
package packetlog
import (
"encoding/xml"
"errors"
"fmt"
"strings"
)
type Message struct {
ID string
From string
To string
Type string
Body string
Subject string
SendTime string
ReceiptID string
ReceiptType string
Consumed bool
Read bool
}
type xmppMessage struct {
XMLName xml.Name `xml:"message"`
ID string `xml:"id,attr"`
From string `xml:"from,attr"`
To string `xml:"to,attr"`
Type string `xml:"type,attr"`
Body string `xml:"body"`
Subject string `xml:"subject"`
Receipt xmppReceipt `xml:"received"`
ISphere xmppISphere `xml:"isphere"`
Consumed *struct{} `xml:"consumed"`
Readed *struct{} `xml:"readed"`
}
type xmppReceipt struct {
ID string `xml:"id,attr"`
Type string `xml:"type,attr"`
}
type xmppISphere struct {
SendTime string `xml:"sendtime,attr"`
}
func ParseMessageLog(plaintext string) (Message, error) {
start := strings.Index(plaintext, "<message")
if start < 0 {
return Message{}, errors.New("message stanza not found")
}
payload := strings.TrimSpace(plaintext[start:])
var parsed xmppMessage
if err := xml.Unmarshal([]byte(payload), &parsed); err != nil {
return Message{}, fmt.Errorf("parse xmpp message: %w", err)
}
if parsed.ID == "" {
return Message{}, errors.New("message id is empty")
}
return Message{
ID: parsed.ID,
From: parsed.From,
To: parsed.To,
Type: parsed.Type,
Body: strings.TrimSpace(parsed.Body),
Subject: strings.TrimSpace(parsed.Subject),
SendTime: parsed.ISphere.SendTime,
ReceiptID: parsed.Receipt.ID,
ReceiptType: parsed.Receipt.Type,
Consumed: parsed.Consumed != nil,
Read: parsed.Readed != nil,
}, nil
}
```
- [ ] **Step 6: Verify PacketReader parser test passes**
Run:
```powershell
go test ./internal/isphere/packetlog -run TestParseMessageLogExtractsXMPPMessage -v
```
Expected: PASS.
- [ ] **Step 7: Run full verification**
Run:
```powershell
go test ./...
go build ./cmd/isphere-mcp
```
Expected: both exit 0.
- [ ] **Step 8: Commit Loop C1**
Run:
```powershell
git add -- internal/isphere/logcodec/codec.go internal/isphere/logcodec/codec_test.go internal/isphere/packetlog/message.go internal/isphere/packetlog/message_test.go docs/superpowers/plans/2026-07-09-stage-c-log-backed-receive-messages-loop.md
git diff --cached --check
git commit -m "feat: parse encrypted isphere packet logs"
```
Expected: commit succeeds.
- [ ] **Step 9: Rewrite next loop before continuing**
After Loop C1 passes, update the `Loop C2` section below with actual package names and parser behavior from the implementation. If Loop C1 exposes a parser limitation, write that limitation into Loop C2 before coding.
---
## Loop C2: Log-backed source abstraction
**Current draft goal:** Wrap `logcodec.DecryptLine` and `packetlog.ParseMessageLog` into a source abstraction that returns normalized messages from a list of encrypted log lines.
**Planned files:**
- Create: `internal/isphere/source.go`
- Create: `internal/isphere/source_test.go`
**Draft interface after Loop C1:**
```go
type Message struct {
ID string
ConversationID string
ConversationType string
SenderID string
ReceiverID string
Text string
Timestamp string
Subject string
ReceiptID string
Read bool
}
type ReceiveMessagesQuery struct {
Limit int
}
type ReceiveMessagesResult struct {
Messages []Message
}
type EncryptedPacketLogSource struct {
Lines []string
}
func (s EncryptedPacketLogSource) ReceiveMessages(ctx context.Context, query ReceiveMessagesQuery) (ReceiveMessagesResult, error)
```
**Rewrite rule before implementation:** Replace this draft with exact code after Loop C1 is committed.
---
## Loop C3: Register `isphere_receive_messages` MCP tool
**Current draft goal:** Add the first business MCP tool after the source abstraction passes tests.
**Planned files:**
- Create: `internal/tools/isphere_read.go`
- Create: `internal/tools/isphere_read_test.go`
- Modify: `internal/mcpserver/server.go`
- Modify: `internal/tools/winhelper_test.go`
**Rewrite rule before implementation:** Update existing tests that currently assert only four helper tools. The new expected surface should be four helper tools plus `isphere_receive_messages`.
---
## Loop C4: Operator verification and docs
**Current draft goal:** Add a verification command or fixture-backed smoke test for `isphere_receive_messages` and update user-facing docs.
**Planned files:**
- Modify: `scripts/verify-go-mcp.ps1`
- Modify: `docs/go-mcp-runbook.md`
- Modify: `docs/current-status-card.md`
**Rewrite rule before implementation:** Only write this loop after C3 confirms the actual MCP response shape.
---
## Self-Review
- Spec coverage: the plan follows the user's cyclic requirement: plan first, implement one loop at a time, update the next loop after each result, and ask only on blockers.
- Placeholder scan: the document has no unresolved placeholder sections. Later loops are explicitly marked as drafts that must be rewritten after prior loop results.
- Type consistency: Loop C1 defines `logcodec.DecryptLine` and `packetlog.ParseMessageLog`; later loops consume those names only after Loop C1 verification.