claw/docs/superpowers/plans/2026-04-04-zhihu-ws-submit-realignment.md

# Zhihu WS Submit Realignment Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Realign Zhihu submit routes to the documented websocket callback model, removing helper-page bootstrap from the mainline while keeping the existing pipe/service contract unchanged.

**Architecture:** The change stays inside the existing submit-path backend selection and websocket protocol flow. Zhihu routes stop choosing `BrowserCallbackBackend` and instead use `WsBrowserBackend` when a real browser websocket is configured, preserving the existing pipe fallback in direct runtime when no websocket URL is available.

**Tech Stack:** Rust, tungstenite websocket client/server, serde_json, cargo test

---

## File Map

- Modify: `src/service/server.rs`
  - Change only the Zhihu route-gated submit-path backend selection
  - Remove Zhihu submit mainline use of `LiveBrowserCallbackHost` / `BrowserCallbackBackend`
  - Keep service submit path on `WsBrowserBackend`
  - Preserve initial request URL derivation for Zhihu routes
- Modify: `src/agent/mod.rs`
  - Change only the Zhihu route-gated submit-path backend selection
  - Remove Zhihu submit mainline use of `LiveBrowserCallbackHost` / `BrowserCallbackBackend`
  - Keep direct runtime pipe fallback when browser websocket URL is absent
- Modify: `tests/agent_runtime_test.rs`
  - Replace helper-page bootstrap regression with direct websocket submit regression
  - Assert no `/sgclaw/browser-helper.html` bootstrap frames are emitted
  - Assert real-page request ownership on follow-up Zhihu actions
- Modify: `src/browser/callback_host.rs`
  - Remove or rewrite the now-wrong red test that preserves Option-B callback-host startup behavior
- Verify: `tests/browser_ws_backend_test.rs`
  - Reuse existing websocket request-url behavior coverage; extend only if the new regression proves insufficient
- Reference: `docs/superpowers/specs/2026-04-04-zhihu-ws-submit-realignment-design.md`

### Task 1: Rewrite the stale submit regression around the real websocket mainline

**Files:**
- Modify: `tests/agent_runtime_test.rs:507-660`
- Test: `tests/agent_runtime_test.rs`

- [ ] **Step 1: Write the failing test**

Rename and rewrite the existing helper-page regression so it asserts the new behavior:

```rust
#[test]
fn production_submit_task_routes_zhihu_through_ws_backend_without_helper_bootstrap() {
    // arrange runtime context and fake browser ws server
    // submit Zhihu hotlist request
    // assert ws frames never contain "/sgclaw/browser-helper.html"
    // assert first action is navigate to https://www.zhihu.com/hot
    // assert follow-up action uses real-page requesturl instead of helper page
}
```

Use the existing fake ws helpers in the file where possible. Do not add localhost callback-host HTTP plumbing to this rewritten test.

- [ ] **Step 2: Run test to verify it fails**

Run: `cargo test --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" production_submit_task_routes_zhihu_through_ws_backend_without_helper_bootstrap --test agent_runtime_test -- --nocapture`

Expected: FAIL because current production code still routes Zhihu submit into `BrowserCallbackBackend` and emits helper-page bootstrap frames.

- [ ] **Step 3: Keep the regression focused**

Before touching production code, confirm the rewritten test checks only these behaviors:

```text
- no callback-host bootstrap frame
- no helper-page URL
- navigate frame still targets https://www.zhihu.com/hot
- follow-up websocket action uses real-page request ownership
```

Do not assert unrelated workflow details beyond what is needed to prove the route correction.

- [ ] **Step 4: Commit the red test**

```bash
git add tests/agent_runtime_test.rs
git commit -m "test: rewrite zhihu submit ws routing regression"
```

### Task 2: Switch service Zhihu submit routes off the callback-host backend

**Files:**
- Modify: `src/service/server.rs:287-328`
- Test: `tests/agent_runtime_test.rs`

- [ ] **Step 1: Write the minimal production change**

Replace only the Zhihu-route callback-host branch with direct websocket backend selection.

Minimal target shape:

```rust
fn browser_backend_for_submit(
    browser_ws_url: &str,
    mac_policy: &MacPolicy,
    request: &SubmitTaskRequest,
) -> Result<Arc<dyn BrowserBackend>, PipeError> {
    if should_use_callback_host_backend(request) {
        return Ok(Arc::new(WsBrowserBackend::new(
            Arc::new(ServiceWsClient::connect(browser_ws_url)?),
            mac_policy.clone(),
            initial_request_url_for_submit_task(request),
        )));
    }

    Ok(Arc::new(WsBrowserBackend::new(
        Arc::new(ServiceWsClient::connect(browser_ws_url)?),
        mac_policy.clone(),
        initial_request_url_for_submit_task(request),
    )))
}
```

After the route-gated branch is removed, simplify further only if the branch becomes redundant without changing non-Zhihu behavior.

- [ ] **Step 2: Run the rewritten regression**

Run: `cargo test --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" production_submit_task_routes_zhihu_through_ws_backend_without_helper_bootstrap --test agent_runtime_test -- --nocapture`

Expected: still FAIL or advance to a later assertion until the direct-runtime path is corrected too.

- [ ] **Step 3: Add or update a service-specific regression if needed**

If the rewritten `agent_runtime_test` does not exercise the service submit path directly, add one narrow service regression before continuing.

Target shape:

```rust
#[test]
fn service_submit_task_routes_zhihu_through_ws_backend_without_helper_bootstrap() {
    // fake browser ws
    // submit Zhihu route through service path
    // assert no helper bootstrap frame
}
```

Run the exact test you end up using:

`cargo test --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" service_submit_task_routes_zhihu_through_ws_backend_without_helper_bootstrap --test <exact test file> -- --nocapture`

Expected: PASS

- [ ] **Step 4: Commit the service-path fix**

```bash
git add src/service/server.rs tests/agent_runtime_test.rs
git commit -m "fix: route zhihu submit through ws backend"
```

### Task 3: Switch direct runtime Zhihu submit routes off the callback-host backend while keeping pipe fallback

**Files:**
- Modify: `src/agent/mod.rs:49-100`
- Test: `tests/agent_runtime_test.rs`

- [ ] **Step 1: Write the minimal production change**

Remove callback-host backend selection from `browser_backend_for_submit(...)`.

Minimal target behavior:

```rust
if let Some(browser_ws_url) = configured_browser_ws_url(context) {
    return Ok(Arc::new(WsBrowserBackend::new(
        Arc::new(ServiceWsClient::connect(&browser_ws_url)?),
        browser_tool.mac_policy().clone(),
        initial_request_url_for_submit_task(request),
    ).with_response_timeout(browser_tool.response_timeout())));
}

Ok(Arc::new(PipeBrowserBackend::from_inner(browser_tool.clone())))
```

If `ServiceWsClient` is not reusable from `src/service/server.rs`, extract the smallest shared websocket client helper into the browser module instead of inventing a new abstraction.

- [ ] **Step 2: Add a focused fallback assertion only if needed**

If the rewritten regression does not cover the direct-runtime no-websocket case, add one small test:

```rust
#[test]
fn production_submit_task_keeps_pipe_fallback_when_browser_ws_url_is_unset() {
    // no SGCLAW_BROWSER_WS_URL
    // blank/no ws config
    // assert no websocket bootstrap attempt occurs
}
```

Only add this test if current coverage is insufficient.

- [ ] **Step 3: Run tests to verify green**

Run: `cargo test --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" production_submit_task_routes_zhihu_through_ws_backend_without_helper_bootstrap --test agent_runtime_test -- --nocapture`

Expected: PASS

If a fallback test was added, run it immediately after and expect PASS.

- [ ] **Step 4: Commit the direct-runtime fix**

```bash
git add src/agent/mod.rs tests/agent_runtime_test.rs
git commit -m "fix: align runtime zhihu submit with ws contract"
```

### Task 4: Reassess stale callback-host regression coverage only if it blocks the approved slice

**Files:**
- Maybe modify: `src/browser/callback_host.rs:793-810`
- Test: `src/browser/callback_host.rs`

- [ ] **Step 1: Check whether the callback-host red test still blocks the approved Option A slice**

Inspect whether this test still preserves rejected Option-B behavior and whether it fails or becomes misleading after Tasks 1-3:

```rust
#[test]
fn live_callback_host_starts_without_bootstrapping_external_helper_page() {
    // inspect before editing
}
```

If the test is unrelated to the approved Zhihu mainline or remains harmless, leave it unchanged in this slice.

- [ ] **Step 2: Remove or rewrite only if required by the changed submit-path behavior**

If the test blocks the approved slice, make the smallest change needed:

- delete it if it exists only to preserve rejected Option B behavior, or
- rewrite it so it no longer asserts callback-host startup as the accepted Zhihu mainline

- [ ] **Step 3: Run focused callback-host tests only if Step 2 changed code**

Run: `cargo test --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" callback_host --lib -- --nocapture`

Expected: PASS

- [ ] **Step 4: Commit only if Step 2 changed code**

```bash
git add src/browser/callback_host.rs
git commit -m "test: clean up stale callback host regression"
```

### Task 5: Run the focused verification sweep

**Files:**
- Verify: `tests/agent_runtime_test.rs`
- Verify: `tests/compat_runtime_test.rs`
- Verify: any directly affected service/browser websocket tests

- [ ] **Step 1: Run submit-path regression coverage**

Run: `cargo test --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" production_submit_task_routes_zhihu_through_ws_backend_without_helper_bootstrap --test agent_runtime_test -- --nocapture`

Expected: PASS

- [ ] **Step 2: Run websocket backend request-url coverage**

Run: `cargo test --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" ws_backend_reuses_last_navigated_url_for_followup_requests --test browser_ws_backend_test -- --nocapture`

Expected: PASS

- [ ] **Step 3: Run Zhihu compat runtime coverage**

Run: `cargo test --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" zhihu --test compat_runtime_test -- --nocapture`

Expected: PASS for the changed submit-path surface or clear, directly related failures only.

- [ ] **Step 4: Run affected service submit regression coverage**

Run the exact service-specific regression from Task 2 if you added one.

Otherwise, run the narrowest existing service submit test that covers `ClientMessage::SubmitTask` handling for browser routes.

Expected: PASS

- [ ] **Step 5: Commit the verified slice**

```bash
git add src/service/server.rs src/agent/mod.rs tests/agent_runtime_test.rs src/browser/callback_host.rs
git commit -m "fix: realign zhihu submit with browser ws callbacks"
```

### Task 6: Run stronger real-browser validation

**Files:**
- Verify live behavior through existing binaries and real config only

- [ ] **Step 1: Start the service with the real config**

Run: `cargo run --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" --bin sg_claw -- --config-path "D:/data/ideaSpace/rust/sgClaw/sgclaw_config.json"`

Expected: service starts without failing at callback-host readiness timeout.

- [ ] **Step 2: Run the client against the started service**

Run: `cargo run --manifest-path "D:/data/ideaSpace/rust/sgClaw/claw-new/Cargo.toml" --bin sg_claw_client`

Expected: for `打开知乎热榜，获取前10条数据，并导出 Excel`, the browser proceeds into real Zhihu page work instead of stalling before page open.

- [ ] **Step 3: Capture the narrow acceptance evidence**

Verify all of the following from logs/frames/observed behavior:

```text
- no callback-host readiness timeout
- no helper-page bootstrap frame
- at least one real-page follow-up browser action after navigate
```

- [ ] **Step 4: Commit only if live verification required code changes**

```bash
git add <only files changed during live-fix follow-up>
git commit -m "fix: tighten zhihu ws submit live validation follow-up"
```

If no further code changes were needed, do not create an extra commit.