添加qiming-rcoder模块
This commit is contained in:
3713
qiming-rcoder/specs/history/agent-abstraction-layer-design.md
Normal file
3713
qiming-rcoder/specs/history/agent-abstraction-layer-design.md
Normal file
File diff suppressed because it is too large
Load Diff
699
qiming-rcoder/specs/history/grpc-migration-rcoder-agentrunner.md
Normal file
699
qiming-rcoder/specs/history/grpc-migration-rcoder-agentrunner.md
Normal file
@@ -0,0 +1,699 @@
|
||||
# rcoder ↔ agent_runner gRPC 通信改造技术方案
|
||||
|
||||
## 1. 背景与动机
|
||||
|
||||
### 1.1 当前架构
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
Client[外部客户端] --> |HTTP + SSE<br/>不可修改| RCoder[rcoder]
|
||||
RCoder --> |HTTP/JSON<br/>reqwest<br/>待改造| AR[agent_runner<br/>容器内]
|
||||
AR --> |SSE Text Stream<br/>待改造| RCoder
|
||||
RCoder --> |SSE| Client
|
||||
```
|
||||
|
||||
### 1.2 当前架构的核心问题
|
||||
|
||||
> [!WARNING]
|
||||
> **核心矛盾**:`rcoder` 本质上只是做**纯转发**,但却需要完整的 HTTP + JSON 序列化/反序列化流程。
|
||||
|
||||
**问题详解**:
|
||||
|
||||
1. ~~**参数重复定义**~~ ✅ **已解决**
|
||||
- `agent_runner/src/model.rs` 现在完全从 `shared_types` 重新导出所有类型
|
||||
- 新增 `CancelNotificationRequestWrapper` 和 `CancelResult` 统一取消操作
|
||||
- 两端共享同一套类型定义,不再需要手工同步
|
||||
|
||||
2. **无意义的序列化开销** ⚠️ 待解决
|
||||
```
|
||||
Client JSON → rcoder 反序列化 → 再序列化 JSON → agent_runner 反序列化
|
||||
```
|
||||
rcoder 收到 JSON 后反序列化为 Rust 结构体,然后又序列化回 JSON 发给 agent_runner,这是**完全冗余的**。
|
||||
|
||||
3. **SSE 文本解析脆弱** ⚠️ 待解决
|
||||
- agent_runner 生成 SSE 事件(`data: {...}\n\n`)
|
||||
- rcoder 接收后需要手工解析 `event:`、`data:` 等文本标记
|
||||
- 再重新构造 SSE 事件发给 Client
|
||||
- 这种文本层面的"拆解-重组"极易出错
|
||||
|
||||
4. **类型约束缺失** ⚠️ 待解决
|
||||
- 两个模块通过 HTTP/JSON 通信,接口兼容性依赖运行时检查
|
||||
- 一方改了字段名或类型,另一方可能静默失败
|
||||
|
||||
**根本原因**:内部模块通信使用了设计给"外部调用"的 HTTP + JSON 协议,不适合紧密耦合的模块间通信场景。
|
||||
|
||||
> [!NOTE]
|
||||
> **进展**:类型共享已通过 `shared_types` 模块实现,gRPC 改造将进一步解决序列化开销和 SSE 解析问题。
|
||||
|
||||
### 1.3 改造边界
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **关键约束**:`rcoder` 对外提供的 HTTP/SSE 接口**不可修改**,本次改造仅涉及 `rcoder ↔ agent_runner` 之间的**内部通信**。
|
||||
|
||||
| 接口 | 改造范围 | 说明 |
|
||||
|------|----------|------|
|
||||
| **rcoder → Client** | ❌ 不改动 | 对外 HTTP API,保持兼容 |
|
||||
| **rcoder ↔ agent_runner** | ✅ 改造目标 | 内部模块通信:HTTP → gRPC |
|
||||
|
||||
### 1.3 当前通信方式问题
|
||||
|
||||
| 问题 | 描述 |
|
||||
|------|------|
|
||||
| **弱类型约束** | `reqwest` 发送 JSON,手工解析 `HttpResult<ChatResponse>`,类型安全依赖运行时检查 |
|
||||
| **SSE 解析脆弱** | `agent_session_notification.rs` 手工解析 SSE 文本流,需处理 `event:`、`data:` 标记 |
|
||||
| **性能开销** | JSON 序列化/反序列化、文本解析带来额外 CPU 开销 |
|
||||
| **双端重复代码** | `ChatRequest`/`ChatResponse` 在 `rcoder` 和 `agent_runner` 分别定义 |
|
||||
|
||||
### 1.4 改造目标
|
||||
|
||||
1. **强类型契约**:使用 Protobuf 定义的 `AgentService` 确保内部接口一致性
|
||||
2. **高效二进制传输**:Protobuf 编码比 JSON 更紧凑、解码更快
|
||||
3. **Server Streaming 替代 SSE**:gRPC 原生流式支持,无需解析文本格式
|
||||
4. **代码共享**:`shared_types` 提供统一的 gRPC client/server 代码
|
||||
|
||||
---
|
||||
|
||||
## 2. 可行性分析
|
||||
|
||||
### 2.1 现有基础设施 ✅
|
||||
|
||||
| 组件 | 状态 | 说明 |
|
||||
|------|------|------|
|
||||
| **tonic 0.14** | ✅ 已添加 | `shared_types/Cargo.toml` 已有依赖 |
|
||||
| **Proto 定义** | ⚠️ 需扩展 | `proto/agent.proto` 定义了 `AgentService`,Attachment 需完善 |
|
||||
| **生成代码** | ✅ 已生成 | `shared_types/src/grpc/agent.rs` 包含 Client/Server |
|
||||
|
||||
### 2.2 Proto 服务定义 (现有)
|
||||
|
||||
```protobuf
|
||||
service AgentService {
|
||||
rpc Chat (ChatRequest) returns (ChatResponse);
|
||||
rpc SubscribeProgress (ProgressRequest) returns (stream ProgressEvent);
|
||||
rpc CancelSession (CancelRequest) returns (CancelResponse);
|
||||
rpc GetStatus (GetStatusRequest) returns (GetStatusResponse);
|
||||
}
|
||||
```
|
||||
|
||||
完全覆盖当前内部 HTTP 接口:
|
||||
|
||||
| agent_runner HTTP 接口 | gRPC 方法 | 类型 |
|
||||
|------------------------|-----------|------|
|
||||
| `POST /chat` | `Chat` | Unary |
|
||||
| `GET /agent/progress/{session_id}` | `SubscribeProgress` | Server Streaming |
|
||||
| `POST /agent/session/cancel` | `CancelSession` | Unary |
|
||||
| `GET /agent/status/{project_id}` | `GetStatus` | Unary |
|
||||
|
||||
### 2.3 改造收益预估
|
||||
|
||||
| 方面 | HTTP/JSON | gRPC/Protobuf | 提升 |
|
||||
|------|-----------|---------------|------|
|
||||
| **编码性能** | ~5μs/msg | ~0.5μs/msg | ~10x |
|
||||
| **消息体积** | ~1KB | ~400B | ~2.5x |
|
||||
| **类型安全** | 运行时 | 编译时 | 显著 |
|
||||
| **流式处理** | 手工解析 SSE | 原生 streaming | 简化 |
|
||||
|
||||
---
|
||||
|
||||
## 3. 架构设计
|
||||
|
||||
### 3.1 目标架构
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "外部接口 (不变)"
|
||||
Client[外部客户端]
|
||||
end
|
||||
|
||||
subgraph "宿主机 - rcoder"
|
||||
RCoder[rcoder<br/>HTTP Server 对外<br/>gRPC Client 对内]
|
||||
end
|
||||
|
||||
subgraph "容器 - agent_runner"
|
||||
AR[agent_runner<br/>gRPC Server 内部<br/>HTTP Health 监控]
|
||||
end
|
||||
|
||||
Client -->|HTTP + SSE<br/>保持不变| RCoder
|
||||
RCoder -->|gRPC unary/stream<br/>内部通信改造| AR
|
||||
|
||||
style Client fill:#e8f5e9
|
||||
style RCoder fill:#e1f5fe
|
||||
style AR fill:#fff3e0
|
||||
```
|
||||
|
||||
### 3.2 端口规划
|
||||
|
||||
| 服务 | 端口 | 用途 | 改动 |
|
||||
|------|------|------|------|
|
||||
| **rcoder HTTP** | 3000 | 对外 HTTP API(Axum) | ❌ 不变 |
|
||||
| **agent_runner gRPC** | 50051 | 容器内 gRPC 服务(Tonic) | ✅ 新增 |
|
||||
| **agent_runner HTTP** | 8086 | 健康检查 `/health`(保留) | ⚠️ 移除业务接口 |
|
||||
|
||||
### 3.3 数据流变化
|
||||
|
||||
**改造前**:
|
||||
```
|
||||
Client → HTTP POST /chat → rcoder → reqwest POST → agent_runner HTTP
|
||||
Client ← SSE ← rcoder ← SSE 文本解析 ← agent_runner SSE
|
||||
```
|
||||
|
||||
**改造后**:
|
||||
```
|
||||
Client → HTTP POST /chat → rcoder → gRPC Chat() → agent_runner gRPC Server
|
||||
Client ← SSE ← rcoder ← gRPC SubscribeProgress() → agent_runner gRPC Server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Proto 定义扩展
|
||||
|
||||
### 4.1 现有 Attachment 问题
|
||||
|
||||
当前 Proto 定义过于简化:
|
||||
|
||||
```protobuf
|
||||
// 现有定义 - 过于简单
|
||||
message Attachment {
|
||||
string name = 1;
|
||||
string kind = 2; // "text", "image" 等,无法区分子结构
|
||||
string content = 3; // 所有数据塞进一个字段
|
||||
string source = 4; // "local", "url" 等
|
||||
optional string language = 5;
|
||||
}
|
||||
```
|
||||
|
||||
Rust 结构体(`shared_types/src/model/attachment.rs`)更丰富:
|
||||
- **4 种附件类型**:Text / Image / Audio / Document
|
||||
- **3 种数据源**:FilePath / Base64 / Url
|
||||
- **类型特有字段**:dimensions、duration、size 等
|
||||
|
||||
### 4.2 扩展 Proto 定义
|
||||
|
||||
```protobuf
|
||||
// === 附件数据源 ===
|
||||
message AttachmentSource {
|
||||
oneof source {
|
||||
string file_path = 1; // 文件路径
|
||||
Base64Data base64 = 2; // Base64 编码数据
|
||||
string url = 3; // URL 链接
|
||||
}
|
||||
}
|
||||
|
||||
message Base64Data {
|
||||
string data = 1;
|
||||
string mime_type = 2;
|
||||
}
|
||||
|
||||
message CancelRequest {
|
||||
string session_id = 1;
|
||||
string reason = 2;
|
||||
}
|
||||
|
||||
message CancelResponse {
|
||||
bool success = 1;
|
||||
CancelResultType result = 2; // 新增:取消结果类型
|
||||
optional string message = 3; // 新增:错误/描述信息
|
||||
}
|
||||
|
||||
// 新增:取消结果枚举(对应 Rust CancelResult)
|
||||
enum CancelResultType {
|
||||
CANCEL_RESULT_SUCCESS = 0;
|
||||
CANCEL_RESULT_FAILED = 1;
|
||||
CANCEL_RESULT_TIMEOUT = 2;
|
||||
}
|
||||
|
||||
// === 附件类型定义 ===
|
||||
message TextAttachment {
|
||||
string id = 1;
|
||||
AttachmentSource source = 2;
|
||||
optional string filename = 3;
|
||||
optional string description = 4;
|
||||
optional string language = 5; // 编程语言(如 "rust", "python")
|
||||
}
|
||||
|
||||
message ImageAttachment {
|
||||
string id = 1;
|
||||
AttachmentSource source = 2;
|
||||
string mime_type = 3; // "image/jpeg", "image/png"
|
||||
optional string filename = 4;
|
||||
optional string description = 5;
|
||||
optional ImageDimensions dimensions = 6;
|
||||
}
|
||||
|
||||
message ImageDimensions {
|
||||
uint32 width = 1;
|
||||
uint32 height = 2;
|
||||
}
|
||||
|
||||
message AudioAttachment {
|
||||
string id = 1;
|
||||
AttachmentSource source = 2;
|
||||
string mime_type = 3; // "audio/mp3", "audio/wav"
|
||||
optional string filename = 4;
|
||||
optional string description = 5;
|
||||
optional double duration = 6; // 时长(秒)
|
||||
}
|
||||
|
||||
message DocumentAttachment {
|
||||
string id = 1;
|
||||
AttachmentSource source = 2;
|
||||
string mime_type = 3; // "application/pdf", "text/plain"
|
||||
optional string filename = 4;
|
||||
optional string description = 5;
|
||||
optional uint64 size = 6; // 文件大小(字节)
|
||||
}
|
||||
|
||||
// === 附件枚举(使用 oneof) ===
|
||||
message Attachment {
|
||||
oneof attachment_type {
|
||||
TextAttachment text = 1;
|
||||
ImageAttachment image = 2;
|
||||
AudioAttachment audio = 3;
|
||||
DocumentAttachment document = 4;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4.3 类型映射表
|
||||
|
||||
| Rust 类型 | Proto 消息 | 说明 |
|
||||
|-----------|------------|------|
|
||||
| `Attachment::Text(TextAttachment)` | `Attachment.text` | oneof 分支 |
|
||||
| `Attachment::Image(ImageAttachment)` | `Attachment.image` | oneof 分支 |
|
||||
| `Attachment::Audio(AudioAttachment)` | `Attachment.audio` | oneof 分支 |
|
||||
| `Attachment::Document(DocumentAttachment)` | `Attachment.document` | oneof 分支 |
|
||||
| `AttachmentSource::FilePath { path }` | `AttachmentSource.file_path` | oneof 分支 |
|
||||
| `AttachmentSource::Base64 { data, mime_type }` | `AttachmentSource.base64` | 嵌套消息 |
|
||||
| `AttachmentSource::Url { url }` | `AttachmentSource.url` | oneof 分支 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 模块改造详情
|
||||
|
||||
### 5.1 shared_types 模块
|
||||
|
||||
#### 5.1.1 更新 proto/agent.proto
|
||||
|
||||
按 4.2 节扩展 Attachment 定义。
|
||||
|
||||
#### 5.1.2 重新生成代码
|
||||
|
||||
```bash
|
||||
cd crates/shared_types
|
||||
cargo build # 触发 build.rs 重新编译 proto
|
||||
```
|
||||
|
||||
### 5.2 agent_runner 模块 (Server 端)
|
||||
|
||||
#### 5.2.1 新增文件结构
|
||||
```
|
||||
crates/agent_runner/src/
|
||||
├── grpc/ # 新增目录
|
||||
│ ├── mod.rs
|
||||
│ └── agent_service_impl.rs # AgentService trait 实现
|
||||
└── main.rs # 修改:启动 gRPC + HTTP 双服务
|
||||
```
|
||||
|
||||
#### 5.2.2 AgentService 实现
|
||||
|
||||
```rust
|
||||
// crates/agent_runner/src/grpc/agent_service_impl.rs
|
||||
use shared_types::grpc::{
|
||||
agent_service_server::AgentService,
|
||||
ChatRequest, ChatResponse, ProgressRequest, ProgressEvent,
|
||||
CancelRequest, CancelResponse, GetStatusRequest, GetStatusResponse,
|
||||
};
|
||||
use tokio_stream::wrappers::ReceiverStream;
|
||||
|
||||
pub struct AgentServiceImpl {
|
||||
// 复用现有 handler 逻辑
|
||||
app_state: Arc<AppState>,
|
||||
}
|
||||
|
||||
#[tonic::async_trait]
|
||||
impl AgentService for AgentServiceImpl {
|
||||
async fn chat(&self, request: Request<ChatRequest>) -> Result<Response<ChatResponse>, Status> {
|
||||
// 复用现有 handler::handle_chat 核心逻辑
|
||||
// Proto 类型 -> 内部类型 -> 调用业务逻辑 -> Proto 响应
|
||||
}
|
||||
|
||||
type SubscribeProgressStream = ReceiverStream<Result<ProgressEvent, Status>>;
|
||||
|
||||
async fn subscribe_progress(
|
||||
&self,
|
||||
request: Request<ProgressRequest>,
|
||||
) -> Result<Response<Self::SubscribeProgressStream>, Status> {
|
||||
// 从内部事件总线读取进度,转换为 ProgressEvent 流
|
||||
// 使用 tokio::sync::mpsc channel
|
||||
}
|
||||
|
||||
async fn cancel_session(&self, request: Request<CancelRequest>) -> Result<Response<CancelResponse>, Status> {
|
||||
// 复用现有取消逻辑
|
||||
}
|
||||
|
||||
async fn get_status(&self, request: Request<GetStatusRequest>) -> Result<Response<GetStatusResponse>, Status> {
|
||||
// 复用现有状态查询逻辑
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 5.2.3 main.rs 修改
|
||||
|
||||
```rust
|
||||
// 双协议启动
|
||||
async fn main() -> Result<()> {
|
||||
// 1. 启动 gRPC Server (业务接口)
|
||||
let grpc_addr = "[::]:50051".parse()?;
|
||||
let grpc_service = AgentServiceServer::new(AgentServiceImpl::new(state.clone()));
|
||||
|
||||
let grpc_server = Server::builder()
|
||||
.add_service(grpc_service)
|
||||
.serve(grpc_addr);
|
||||
|
||||
// 2. 启动 HTTP Server (仅保留 /health)
|
||||
let http_router = Router::new()
|
||||
.route("/health", get(health_check));
|
||||
let http_addr = "0.0.0.0:8086".parse()?;
|
||||
let http_server = axum::serve(TcpListener::bind(http_addr).await?, http_router);
|
||||
|
||||
// 3. 并行运行
|
||||
info!("🚀 agent_runner 启动: gRPC=50051, HTTP Health=8086");
|
||||
tokio::select! {
|
||||
r = grpc_server => r?,
|
||||
r = http_server => r?,
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### 5.3 rcoder 模块 (Client 端)
|
||||
|
||||
> [!IMPORTANT]
|
||||
> rcoder 对外 HTTP 接口**完全保持不变**,仅修改与 agent_runner 通信的内部实现。
|
||||
|
||||
#### 5.3.1 新增文件
|
||||
```
|
||||
crates/rcoder/src/
|
||||
├── grpc/ # 新增目录
|
||||
│ ├── mod.rs
|
||||
│ ├── channel_pool.rs # gRPC Channel 连接池
|
||||
│ └── converters.rs # HTTP ↔ gRPC 类型转换
|
||||
└── handler/
|
||||
├── chat_handler.rs # 内部改用 gRPC client
|
||||
└── agent_session_notification.rs # 内部调用 SubscribeProgress
|
||||
```
|
||||
|
||||
#### 5.3.2 chat_handler.rs 改造
|
||||
|
||||
```rust
|
||||
// 改造前 (内部使用 HTTP)
|
||||
async fn forward_request_to_container_service(
|
||||
request: &ChatRequest,
|
||||
container_info: &ContainerBasicInfo,
|
||||
) -> Result<HttpResult<ChatResponse>, AppError> {
|
||||
let client = reqwest::Client::new();
|
||||
let chat_url = format!("{}/chat", container_info.service_url);
|
||||
let response = client.post(&chat_url).json(request).send().await?;
|
||||
// ... 手工解析 JSON
|
||||
}
|
||||
|
||||
// 改造后 (内部使用 gRPC)
|
||||
async fn forward_request_to_container_service(
|
||||
request: &ChatRequest,
|
||||
container_info: &ContainerBasicInfo,
|
||||
channel_pool: &GrpcChannelPool,
|
||||
) -> Result<HttpResult<ChatResponse>, AppError> {
|
||||
// 构建 gRPC 地址
|
||||
let grpc_addr = format!("http://{}:50051", container_info.ip_address);
|
||||
let channel = channel_pool.get_or_create_channel(&grpc_addr).await?;
|
||||
|
||||
// 类型转换:HTTP ChatRequest -> gRPC ChatRequest
|
||||
let grpc_request = converters::http_to_grpc_chat_request(request);
|
||||
|
||||
// gRPC 调用
|
||||
let mut client = AgentServiceClient::new(channel);
|
||||
let response = client.chat(grpc_request).await
|
||||
.map_err(|e| AppError::internal_server_error(&format!("gRPC error: {}", e)))?;
|
||||
|
||||
// 类型转换:gRPC ChatResponse -> HTTP ChatResponse
|
||||
converters::grpc_to_http_result(response.into_inner())
|
||||
}
|
||||
```
|
||||
|
||||
#### 5.3.3 agent_session_notification.rs 改造
|
||||
|
||||
```rust
|
||||
// 改造前 (SSE 代理解析)
|
||||
async fn create_sse_proxy_stream(
|
||||
sse_url: String,
|
||||
session_id: String,
|
||||
) -> impl Stream<Item = Result<Event, Infallible>> {
|
||||
// 手工解析 SSE 文本流 "event:", "data:" 等
|
||||
}
|
||||
|
||||
// 改造后 (gRPC Streaming → SSE 转发)
|
||||
async fn create_sse_from_grpc_stream(
|
||||
container_addr: String,
|
||||
session_id: String,
|
||||
channel_pool: &GrpcChannelPool,
|
||||
) -> impl Stream<Item = Result<Event, Infallible>> {
|
||||
let channel = channel_pool.get_or_create_channel(&container_addr).await.unwrap();
|
||||
let mut client = AgentServiceClient::new(channel);
|
||||
|
||||
let request = ProgressRequest { session_id };
|
||||
let grpc_stream = client.subscribe_progress(request).await.unwrap().into_inner();
|
||||
|
||||
// 将 gRPC ProgressEvent 转换为 Axum SSE Event (对外格式不变)
|
||||
grpc_stream.map(|result| {
|
||||
match result {
|
||||
Ok(event) => Ok(converters::progress_to_sse_event(event)),
|
||||
Err(status) => Ok(Event::default().data(format!("error: {}", status))),
|
||||
}
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. 类型转换层
|
||||
|
||||
由于 rcoder 对外 HTTP 格式不变,需要在 `rcoder` 中实现类型转换:
|
||||
|
||||
```rust
|
||||
// crates/rcoder/src/grpc/converters.rs
|
||||
|
||||
/// HTTP ChatRequest -> gRPC ChatRequest
|
||||
pub fn http_to_grpc_chat_request(
|
||||
http_req: &crate::handler::ChatRequest,
|
||||
) -> shared_types::grpc::ChatRequest {
|
||||
shared_types::grpc::ChatRequest {
|
||||
project_id: http_req.project_id.clone().unwrap_or_default(),
|
||||
session_id: http_req.session_id.clone().unwrap_or_default(),
|
||||
prompt: http_req.prompt.clone(),
|
||||
model_config: http_req.model_provider.as_ref().map(model_provider_to_grpc),
|
||||
attachments: http_req.attachments.iter().map(attachment_to_grpc).collect(),
|
||||
request_id: http_req.request_id.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Rust Attachment -> gRPC Attachment
|
||||
fn attachment_to_grpc(att: &shared_types::Attachment) -> shared_types::grpc::Attachment {
|
||||
// 使用 oneof 分支匹配
|
||||
match att {
|
||||
Attachment::Text(t) => { /* 构建 TextAttachment */ },
|
||||
Attachment::Image(i) => { /* 构建 ImageAttachment */ },
|
||||
// ...
|
||||
}
|
||||
}
|
||||
|
||||
/// gRPC ProgressEvent -> Axum SSE Event (保持对外格式不变)
|
||||
pub fn progress_to_sse_event(event: shared_types::grpc::ProgressEvent) -> axum::sse::Event {
|
||||
// 优先使用 json_payload 字段保持兼容
|
||||
// 或根据 oneof event 字段构建 SSE 事件
|
||||
Event::default()
|
||||
.event("message")
|
||||
.data(event.json_payload)
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 实施计划
|
||||
|
||||
### 7.1 阶段划分
|
||||
|
||||
| 阶段 | 内容 | 工作量 | 风险 |
|
||||
|------|------|--------|------|
|
||||
| **Phase 1** | Proto Attachment 扩展 | 0.5天 | 低 |
|
||||
| **Phase 2** | agent_runner gRPC Server | 2天 | 中 |
|
||||
| **Phase 3** | rcoder gRPC Client (内部改造) | 2天 | 中 |
|
||||
| **Phase 4** | 集成测试与清理 | 1天 | 低 |
|
||||
|
||||
### 7.2 Phase 1: Proto Attachment 扩展
|
||||
|
||||
- [ ] 更新 `shared_types/proto/agent.proto`,添加完整 Attachment 定义
|
||||
- [ ] 运行 `cargo build -p shared_types` 验证生成代码
|
||||
- [ ] 检查生成的 Rust 类型与现有类型的兼容性
|
||||
|
||||
### 7.3 Phase 2: agent_runner gRPC Server
|
||||
|
||||
- [ ] 创建 `src/grpc/mod.rs` 和 `agent_service_impl.rs`
|
||||
- [ ] 实现 `AgentService` trait 四个方法
|
||||
- [ ] 修改 `main.rs` 启动 gRPC + HTTP 双服务
|
||||
- [ ] 移除 agent_runner 中的业务 HTTP 接口 (`/chat`, `/agent/*`)
|
||||
- [ ] 保留 `/health` 健康检查接口
|
||||
- [ ] 单元测试各 RPC 方法
|
||||
|
||||
### 7.4 Phase 3: rcoder gRPC Client
|
||||
|
||||
- [ ] 创建 `src/grpc/channel_pool.rs`
|
||||
- [ ] 创建 `src/grpc/converters.rs` 实现类型转换
|
||||
- [ ] 在 `AppState` 中添加 `GrpcChannelPool`
|
||||
- [ ] 改造 `chat_handler.rs` 内部实现(对外接口保持不变)
|
||||
- [ ] 改造 `agent_session_notification.rs` 内部实现
|
||||
- [ ] 移除 reqwest SSE 解析代码
|
||||
|
||||
### 7.5 Phase 4: 集成测试与清理
|
||||
|
||||
- [ ] 端到端测试完整流程(Client → rcoder HTTP → gRPC → agent_runner)
|
||||
- [ ] 验证对外 HTTP API 行为完全不变
|
||||
- [ ] 验证 SSE 输出格式与改造前一致
|
||||
- [ ] 更新文档
|
||||
|
||||
---
|
||||
|
||||
## 8. 验证计划
|
||||
|
||||
### 8.1 单元测试
|
||||
|
||||
```bash
|
||||
# agent_runner gRPC 服务测试
|
||||
cargo test -p agent_runner --lib grpc
|
||||
|
||||
# rcoder gRPC 客户端测试
|
||||
cargo test -p rcoder --lib grpc
|
||||
```
|
||||
|
||||
### 8.2 gRPC 接口测试
|
||||
|
||||
```bash
|
||||
# 1. 启动容器化 agent_runner
|
||||
docker run -p 50051:50051 -p 8086:8086 agent-runner:latest
|
||||
|
||||
# 2. 使用 grpcurl 测试 Chat (直接调用 gRPC)
|
||||
grpcurl -plaintext -d '{"project_id":"test","session_id":"s1","prompt":"hello"}' \
|
||||
localhost:50051 agent.AgentService/Chat
|
||||
|
||||
# 3. 使用 grpcurl 测试 SubscribeProgress
|
||||
grpcurl -plaintext -d '{"session_id":"s1"}' \
|
||||
localhost:50051 agent.AgentService/SubscribeProgress
|
||||
```
|
||||
|
||||
### 8.3 端到端测试(验证对外接口不变)
|
||||
|
||||
```bash
|
||||
# 启动 rcoder
|
||||
cargo run -p rcoder
|
||||
|
||||
# 测试聊天 - 对外 HTTP 接口不变
|
||||
curl -X POST http://localhost:3000/chat \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"prompt":"test"}'
|
||||
|
||||
# 测试 SSE - 对外格式不变
|
||||
curl -N http://localhost:3000/agent/progress/{session_id}
|
||||
```
|
||||
|
||||
### 8.4 回归测试
|
||||
|
||||
确保以下场景行为与改造前一致:
|
||||
|
||||
| 测试用例 | 验证点 |
|
||||
|----------|--------|
|
||||
| 正常聊天请求 | 返回 `HttpResult<ChatResponse>` |
|
||||
| 带附件的请求 | 附件正确传递到 agent_runner |
|
||||
| SSE 订阅 | 事件格式、字段与改造前一致 |
|
||||
| 会话取消 | 取消成功,SSE 流关闭 |
|
||||
| 健康检查 | `GET /health` 返回 200 |
|
||||
|
||||
---
|
||||
|
||||
## 9. 回滚策略
|
||||
|
||||
### 9.1 Feature Flag 支持
|
||||
|
||||
```rust
|
||||
// rcoder/Cargo.toml
|
||||
[features]
|
||||
default = ["grpc-internal"]
|
||||
grpc-internal = []
|
||||
http-legacy = []
|
||||
```
|
||||
|
||||
```rust
|
||||
// chat_handler.rs
|
||||
#[cfg(feature = "grpc-internal")]
|
||||
async fn forward_request(...) { /* gRPC 实现 */ }
|
||||
|
||||
#[cfg(feature = "http-legacy")]
|
||||
async fn forward_request(...) { /* HTTP 实现 */ }
|
||||
```
|
||||
|
||||
### 9.2 回滚方案
|
||||
|
||||
1. Git revert 到改造前 commit
|
||||
2. 或切换 feature:`cargo build --no-default-features --features http-legacy`
|
||||
|
||||
---
|
||||
|
||||
## 10. 风险与缓解
|
||||
|
||||
| 风险 | 影响 | 缓解措施 |
|
||||
|------|------|----------|
|
||||
| gRPC 连接失败 | 内部请求中断 | 增加连接重试、健康检查 |
|
||||
| Proto 定义变更 | 内部不兼容 | 严格 proto 版本管理 |
|
||||
| 容器网络问题 | gRPC 不通 | 保留 HTTP health check |
|
||||
| 流式中断 | 进度丢失 | 客户端重连机制 |
|
||||
| 类型转换错误 | 数据丢失 | 完善单元测试覆盖 |
|
||||
|
||||
---
|
||||
|
||||
## 附录 A: 文件改动清单
|
||||
|
||||
| 模块 | 文件 | 改动类型 | 说明 |
|
||||
|------|------|----------|------|
|
||||
| shared_types | `proto/agent.proto` | 修改 | 扩展 Attachment 定义 |
|
||||
| agent_runner | `src/grpc/mod.rs` | 新增 | gRPC 模块入口 |
|
||||
| agent_runner | `src/grpc/agent_service_impl.rs` | 新增 | AgentService 实现 |
|
||||
| agent_runner | `src/main.rs` | 修改 | 双协议启动 |
|
||||
| agent_runner | `src/handler/*.rs` | 删除 | 移除业务 HTTP 接口 |
|
||||
| rcoder | `src/grpc/mod.rs` | 新增 | gRPC 模块入口 |
|
||||
| rcoder | `src/grpc/channel_pool.rs` | 新增 | 连接池管理 |
|
||||
| rcoder | `src/grpc/converters.rs` | 新增 | 类型转换 |
|
||||
| rcoder | `src/handler/chat_handler.rs` | 修改 | 内部改用 gRPC |
|
||||
| rcoder | `src/handler/agent_session_notification.rs` | 修改 | gRPC Streaming |
|
||||
| rcoder | `src/router.rs` | 不变 | 对外路由不变 |
|
||||
|
||||
---
|
||||
|
||||
## 附录 B: 依赖版本
|
||||
|
||||
```toml
|
||||
# shared_types/Cargo.toml
|
||||
tonic = { version = "0.14.2", features = ["tls-native-roots"] }
|
||||
tonic-prost = "0.14"
|
||||
prost = "0.14"
|
||||
prost-types = "0.14"
|
||||
|
||||
# build-dependencies
|
||||
tonic-prost-build = "0.14"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**文档版本**: v1.2
|
||||
**创建日期**: 2025-12-05
|
||||
**更新日期**: 2025-12-06
|
||||
**变更记录**:
|
||||
- v1.2 (2025-12-06): 更新类型共享现状(已通过 shared_types 实现),新增 CancelResult 类型定义
|
||||
- v1.1 (2025-12-05): 明确改造边界,扩展 Attachment Proto 定义
|
||||
- v1.0 (2025-12-05): 初始版本
|
||||
710
qiming-rcoder/specs/history/multi-docker-image-design.md
Normal file
710
qiming-rcoder/specs/history/multi-docker-image-design.md
Normal file
@@ -0,0 +1,710 @@
|
||||
# RCoder 双 Docker 镜像配置技术设计文档
|
||||
|
||||
## 📋 概述
|
||||
|
||||
本文档描述了 RCoder 系统中双 Docker 镜像配置的设计方案,支持在动态创建容器时指定 RCoder 或 AgentRunner 服务类型,以支持当前功能和未来新功能的开发。
|
||||
|
||||
### 背景
|
||||
|
||||
当前 RCoder 系统使用 `registry.yichamao.com/rcoder` 镜像。为了支持新功能开发,需要引入 `registry.yichamao.com/rcoder-agent-runner` 镜像,同时保持现有功能的稳定性。需要设计一个简化的双镜像配置系统。
|
||||
|
||||
### 目标
|
||||
|
||||
1. **支持两种服务类型**: rcoder (当前使用) 和 agent-runner (新功能)
|
||||
2. **保持架构兼容性**: 支持 ARM64/AMD64 多架构
|
||||
3. **灵活配置**: 支持默认镜像、服务特定镜像、项目级镜像覆盖
|
||||
4. **向后兼容**: 不破坏现有配置和功能,默认使用 rcoder 服务
|
||||
5. **简化实现**: 降低复杂度,便于维护和扩展
|
||||
|
||||
## 🏗️ 整体架构设计
|
||||
|
||||
### 配置层级结构
|
||||
|
||||
```
|
||||
全局默认镜像配置
|
||||
↓
|
||||
服务类型特定配置 (rcoder/agent-runner/specialized-tools)
|
||||
↓
|
||||
项目级镜像覆盖 (可选)
|
||||
↓
|
||||
运行时镜像选择 (最终使用的镜像)
|
||||
```
|
||||
|
||||
### 核心组件
|
||||
|
||||
1. **ServiceType**: 服务类型枚举
|
||||
2. **MultiImageConfig**: 多镜像配置结构
|
||||
3. **ImageSelector**: 镜像选择器
|
||||
4. **ImageRegistry**: 镜像注册表
|
||||
|
||||
## 📐 详细设计
|
||||
|
||||
### 1. 服务类型定义
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Hash)]
|
||||
pub enum ServiceType {
|
||||
/// 标准的 rcoder 服务 (当前默认使用的服务)
|
||||
RCoder,
|
||||
/// Agent Runner 服务 (新功能服务,后续开发使用)
|
||||
AgentRunner,
|
||||
}
|
||||
|
||||
impl ServiceType {
|
||||
pub fn as_str(&self) -> &str {
|
||||
match self {
|
||||
ServiceType::RCoder => "rcoder",
|
||||
ServiceType::AgentRunner => "agent-runner",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn from_str(s: &str) -> Self {
|
||||
match s {
|
||||
"rcoder" => ServiceType::RCoder,
|
||||
"agent-runner" => ServiceType::AgentRunner,
|
||||
_ => {
|
||||
tracing::warn!("未知的服务类型 '{}',使用默认的 RCoder 服务", s);
|
||||
ServiceType::RCoder
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 获取服务描述
|
||||
pub fn description(&self) -> &str {
|
||||
match self {
|
||||
ServiceType::RCoder => "标准 RCoder 服务,提供完整的 AI 开发功能",
|
||||
ServiceType::AgentRunner => "Agent Runner 服务,专注于代理运行和执行",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 注意:移除了 Default trait,强制要求明确指定服务类型
|
||||
```
|
||||
|
||||
### 2. 服务镜像配置
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ServiceImageConfig {
|
||||
/// 服务类型
|
||||
pub service_type: ServiceType,
|
||||
/// 通用镜像(优先级最高)
|
||||
pub image: Option<String>,
|
||||
/// ARM64 架构专用镜像
|
||||
pub arm64_image: Option<String>,
|
||||
/// AMD64 架构专用镜像
|
||||
pub amd64_image: Option<String>,
|
||||
/// 默认回退镜像
|
||||
pub default_image: Option<String>,
|
||||
/// 镜像标签前缀(用于自动构建镜像名称)
|
||||
pub image_tag_prefix: Option<String>,
|
||||
/// 是否启用该服务类型
|
||||
pub enabled: bool,
|
||||
/// 服务特定的环境变量
|
||||
pub environment: HashMap<String, String>,
|
||||
/// 服务特定的挂载点
|
||||
pub mounts: Vec<ServiceMountConfig>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ServiceMountConfig {
|
||||
/// 容器内路径
|
||||
pub container_path: String,
|
||||
/// 宿主机路径(支持变量替换)
|
||||
pub host_path: String,
|
||||
/// 是否只读
|
||||
pub read_only: bool,
|
||||
/// 挂载类型
|
||||
pub mount_type: String, // "bind", "volume"
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 多镜像配置结构
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct MultiImageConfig {
|
||||
/// 全局默认镜像配置
|
||||
pub global_defaults: GlobalImageDefaults,
|
||||
/// 各服务类型的镜像配置
|
||||
pub services: HashMap<String, ServiceImageConfig>,
|
||||
/// 镜像选择策略
|
||||
pub selection_strategy: ImageSelectionStrategy,
|
||||
/// 镜像缓存配置
|
||||
pub cache_config: ImageCacheConfig,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct GlobalImageDefaults {
|
||||
/// 通用默认镜像
|
||||
pub image: Option<String>,
|
||||
/// 默认 ARM64 镜像
|
||||
pub arm64_image: Option<String>,
|
||||
/// 默认 AMD64 镜像
|
||||
pub amd64_image: Option<String>,
|
||||
/// 默认回退镜像
|
||||
pub default_image: Option<String>,
|
||||
/// 镜像仓库前缀
|
||||
pub registry_prefix: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub enum ImageSelectionStrategy {
|
||||
/// 仅使用服务特定配置(强制明确指定服务类型)
|
||||
ServiceOnly,
|
||||
}
|
||||
|
||||
impl Default for ImageSelectionStrategy {
|
||||
fn default() -> Self {
|
||||
ImageSelectionStrategy::ServiceOnly
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ImageCacheConfig {
|
||||
/// 是否启用镜像缓存
|
||||
pub enabled: bool,
|
||||
/// 缓存过期时间(秒)
|
||||
pub ttl_seconds: u64,
|
||||
/// 最大缓存条目数
|
||||
pub max_entries: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### 4. 配置文件结构
|
||||
|
||||
```yaml
|
||||
# Docker 双镜像配置 - 简化版本(仅支持 RCoder 和 AgentRunner)
|
||||
# 注意:创建容器时必须明确指定服务类型,不允许默认值
|
||||
docker_config:
|
||||
# 全局默认配置
|
||||
global_defaults:
|
||||
# 默认镜像仓库前缀
|
||||
registry_prefix: "registry.yichamao.com"
|
||||
|
||||
# 全局默认镜像配置
|
||||
image: null # 留空使用服务特定配置
|
||||
arm64_image: "registry.yichamao.com/default:latest-arm64"
|
||||
amd64_image: "registry.yichamao.com/default:latest-amd64"
|
||||
default_image: "registry.yichamao.com/default:latest"
|
||||
|
||||
# 镜像选择策略
|
||||
selection_strategy: "ServiceOnly" # 仅使用服务特定配置,强制明确指定
|
||||
|
||||
# 各服务类型的配置
|
||||
services:
|
||||
# 标准 RCoder 服务配置 (当前项目使用)
|
||||
rcoder:
|
||||
service_type: "rcoder"
|
||||
image: null # 使用架构特定镜像
|
||||
arm64_image: "registry.yichamao.com/rcoder:latest-arm64"
|
||||
amd64_image: "registry.yichamao.com/rcoder:latest-amd64"
|
||||
default_image: "registry.yichamao.com/rcoder:latest"
|
||||
image_tag_prefix: "rcoder"
|
||||
enabled: true # 当前启用
|
||||
environment:
|
||||
RUST_LOG: "info"
|
||||
SERVICE_MODE: "full"
|
||||
API_PORT: "8086"
|
||||
mounts:
|
||||
- container_path: "/app/project_workspace"
|
||||
host_path: "./project_workspace"
|
||||
read_only: false
|
||||
mount_type: "bind"
|
||||
|
||||
# Agent Runner 服务配置 (新功能,后续开发使用)
|
||||
agent-runner:
|
||||
service_type: "agent-runner"
|
||||
image: null # 使用架构特定镜像
|
||||
arm64_image: "registry.yichamao.com/rcoder-agent-runner:latest-arm64"
|
||||
amd64_image: "registry.yichamao.com/rcoder-agent-runner:latest-amd64"
|
||||
default_image: "registry.yichamao.com/rcoder-agent-runner:latest"
|
||||
image_tag_prefix: "rcoder-agent-runner"
|
||||
enabled: false # 默认禁用,等待新功能开发
|
||||
environment:
|
||||
RUST_LOG: "debug"
|
||||
SERVICE_MODE: "agent-only"
|
||||
AGENT_PORT: "8086"
|
||||
mounts:
|
||||
- container_path: "/app/workspace"
|
||||
host_path: "./project_workspace/{project_id}"
|
||||
read_only: false
|
||||
mount_type: "bind"
|
||||
- container_path: "/app/models"
|
||||
host_path: "./models"
|
||||
read_only: true
|
||||
mount_type: "bind"
|
||||
|
||||
# 镜像缓存配置
|
||||
cache_config:
|
||||
enabled: true
|
||||
ttl_seconds: 3600 # 1小时
|
||||
max_entries: 50 # 减少缓存条目数,因为只有2个服务
|
||||
|
||||
# 其他现有配置保持不变
|
||||
network_mode: "bridge"
|
||||
work_dir: "/app"
|
||||
auto_cleanup: true
|
||||
container_ttl_seconds: 3600
|
||||
```
|
||||
|
||||
### 5. 镜像选择器实现
|
||||
|
||||
```rust
|
||||
pub struct ImageSelector {
|
||||
config: MultiImageConfig,
|
||||
cache: Arc<RwLock<HashMap<String, CachedImageInfo>>>,
|
||||
platform: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CachedImageInfo {
|
||||
pub image_name: String,
|
||||
pub service_type: ServiceType,
|
||||
pub platform: String,
|
||||
pub cached_at: std::time::SystemTime,
|
||||
}
|
||||
|
||||
impl ImageSelector {
|
||||
pub fn new(config: MultiImageConfig) -> Self {
|
||||
let platform = crate::utils::DockerUtils::get_optimal_platform();
|
||||
Self {
|
||||
config,
|
||||
cache: Arc::new(RwLock::new(HashMap::new())),
|
||||
platform,
|
||||
}
|
||||
}
|
||||
|
||||
/// 根据服务类型和项目配置选择镜像
|
||||
/// 注意:service_type 不能为空,必须明确指定
|
||||
pub fn select_image(
|
||||
&self,
|
||||
service_type: &ServiceType,
|
||||
project_overrides: Option<&ProjectImageOverrides>,
|
||||
) -> DockerResult<String> {
|
||||
// 强制验证:service_type 必须明确指定
|
||||
if !self.is_service_enabled(service_type) {
|
||||
return Err(DockerError::ConfigurationError(
|
||||
format!("服务类型 '{}' 未启用或配置不存在", service_type.as_str())
|
||||
));
|
||||
}
|
||||
|
||||
let cache_key = self.build_cache_key(service_type, project_overrides);
|
||||
|
||||
// 检查缓存
|
||||
if let Some(cached) = self.get_from_cache(&cache_key) {
|
||||
return Ok(cached.image_name);
|
||||
}
|
||||
|
||||
// 强制使用 ServiceOnly 策略:仅使用服务特定配置
|
||||
let image_name = self.select_service_only(service_type, project_overrides)?;
|
||||
|
||||
// 缓存结果
|
||||
self.cache_image_info(&cache_key, &image_name, service_type);
|
||||
|
||||
Ok(image_name)
|
||||
}
|
||||
|
||||
/// 检查服务是否已启用和配置
|
||||
fn is_service_enabled(&self, service_type: &ServiceType) -> bool {
|
||||
let service_key = service_type.as_str();
|
||||
if let Some(service_config) = self.config.services.get(service_key) {
|
||||
service_config.enabled
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
fn select_service_first(
|
||||
&self,
|
||||
service_type: &ServiceType,
|
||||
project_overrides: Option<&ProjectImageOverrides>,
|
||||
) -> DockerResult<String> {
|
||||
let service_key = service_type.as_str();
|
||||
|
||||
// 1. 检查项目级覆盖
|
||||
if let Some(overrides) = project_overrides {
|
||||
if let Some(project_image) = self.get_project_override_image(overrides, service_type) {
|
||||
return Ok(project_image);
|
||||
}
|
||||
}
|
||||
|
||||
// 2. 检查服务特定配置
|
||||
if let Some(service_config) = self.config.services.get(service_key) {
|
||||
if !service_config.enabled {
|
||||
return Err(DockerError::ConfigurationError(
|
||||
format!("服务类型 {} 未启用", service_key)
|
||||
));
|
||||
}
|
||||
|
||||
if let Some(image) = self.select_service_image(service_config) {
|
||||
return Ok(image);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. 回退到全局默认配置
|
||||
self.select_global_default_image()
|
||||
}
|
||||
|
||||
fn select_service_image(&self, config: &ServiceImageConfig) -> Option<String> {
|
||||
// 1. 优先使用通用镜像
|
||||
if let Some(image) = &config.image {
|
||||
return Some(image.clone());
|
||||
}
|
||||
|
||||
// 2. 根据架构选择特定镜像
|
||||
match self.platform.as_str() {
|
||||
"linux/arm64" => {
|
||||
config.arm64_image
|
||||
.clone()
|
||||
.or_else(|| config.default_image.clone())
|
||||
}
|
||||
"linux/amd64" => {
|
||||
config.amd64_image
|
||||
.clone()
|
||||
.or_else(|| config.default_image.clone())
|
||||
}
|
||||
_ => config.default_image.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
fn select_global_default_image(&self) -> DockerResult<String> {
|
||||
let defaults = &self.config.global_defaults;
|
||||
|
||||
// 1. 优先使用全局通用镜像
|
||||
if let Some(image) = &defaults.image {
|
||||
return Ok(image.clone());
|
||||
}
|
||||
|
||||
// 2. 根据架构选择全局默认镜像
|
||||
let image = match self.platform.as_str() {
|
||||
"linux/arm64" => {
|
||||
defaults.arm64_image
|
||||
.clone()
|
||||
.or_else(|| defaults.default_image.clone())
|
||||
}
|
||||
"linux/amd64" => {
|
||||
defaults.amd64_image
|
||||
.clone()
|
||||
.or_else(|| defaults.default_image.clone())
|
||||
}
|
||||
_ => defaults.default_image.clone(),
|
||||
};
|
||||
|
||||
image.ok_or_else(|| {
|
||||
DockerError::ConfigurationError(
|
||||
"无法找到适合的默认镜像配置".to_string()
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
fn build_cache_key(
|
||||
&self,
|
||||
service_type: &ServiceType,
|
||||
project_overrides: Option<&ProjectImageOverrides>,
|
||||
) -> String {
|
||||
let base_key = format!("{}:{}", service_type.as_str(), self.platform);
|
||||
|
||||
if let Some(overrides) = project_overrides {
|
||||
format!("{}:overrides:{}", base_key, overrides.hash_key())
|
||||
} else {
|
||||
base_key
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 项目级镜像覆盖配置
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ProjectImageOverrides {
|
||||
/// 项目特定的镜像配置
|
||||
pub images: HashMap<String, String>,
|
||||
/// 启用的服务类型
|
||||
pub enabled_services: Vec<String>,
|
||||
/// 项目特定的环境变量
|
||||
pub environment: HashMap<String, String>,
|
||||
}
|
||||
|
||||
impl ProjectImageOverrides {
|
||||
pub fn hash_key(&self) -> String {
|
||||
use std::collections::hash_map::DefaultHasher;
|
||||
use std::hash::{Hash, Hasher};
|
||||
|
||||
let mut hasher = DefaultHasher::new();
|
||||
|
||||
// 哈希镜像配置
|
||||
for (key, value) in &self.images {
|
||||
key.hash(&mut hasher);
|
||||
value.hash(&mut hasher);
|
||||
}
|
||||
|
||||
// 哈希启用的服务
|
||||
for service in &self.enabled_services {
|
||||
service.hash(&mut hasher);
|
||||
}
|
||||
|
||||
format!("{:x}", hasher.finish())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 🔧 实现建议
|
||||
|
||||
### 1. 配置文件兼容性
|
||||
|
||||
为保持向后兼容,支持两种配置方式:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct DockerConfig {
|
||||
/// 新的多镜像配置(优先)
|
||||
#[serde(default)]
|
||||
pub multi_image_config: Option<MultiImageConfig>,
|
||||
|
||||
/// 传统单一镜像配置(向后兼容)
|
||||
pub image: Option<String>,
|
||||
pub arm64_image: Option<String>,
|
||||
pub amd64_image: Option<String>,
|
||||
pub default_image: Option<String>,
|
||||
|
||||
// ... 其他现有字段
|
||||
}
|
||||
|
||||
impl DockerConfig {
|
||||
/// 获取多镜像配置,如果未配置则使用传统配置创建默认配置
|
||||
pub fn get_multi_image_config(&self) -> MultiImageConfig {
|
||||
if let Some(ref config) = self.multi_image_config {
|
||||
config.clone()
|
||||
} else {
|
||||
// 从传统配置创建默认多镜像配置
|
||||
self.create_legacy_multi_config()
|
||||
}
|
||||
}
|
||||
|
||||
fn create_legacy_multi_config(&self) -> MultiImageConfig {
|
||||
MultiImageConfig {
|
||||
default_service_type: ServiceType::RCoder,
|
||||
global_defaults: GlobalImageDefaults {
|
||||
image: self.image.clone(),
|
||||
arm64_image: self.arm64_image.clone(),
|
||||
amd64_image: self.amd64_image.clone(),
|
||||
default_image: self.default_image.clone(),
|
||||
registry_prefix: Some("registry.yichamao.com".to_string()),
|
||||
},
|
||||
services: {
|
||||
let mut services = HashMap::new();
|
||||
services.insert(
|
||||
"rcoder".to_string(),
|
||||
ServiceImageConfig {
|
||||
service_type: ServiceType::RCoder,
|
||||
image: self.image.clone(),
|
||||
arm64_image: self.arm64_image.clone(),
|
||||
amd64_image: self.amd64_image.clone(),
|
||||
default_image: self.default_image.clone(),
|
||||
image_tag_prefix: Some("rcoder".to_string()),
|
||||
enabled: true,
|
||||
environment: HashMap::new(),
|
||||
mounts: Vec::new(),
|
||||
},
|
||||
);
|
||||
services
|
||||
},
|
||||
selection_strategy: ImageSelectionStrategy::ServiceFirst,
|
||||
cache_config: ImageCacheConfig {
|
||||
enabled: true,
|
||||
ttl_seconds: 3600,
|
||||
max_entries: 100,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 2. 容器创建接口更新
|
||||
|
||||
```rust
|
||||
impl DockerManager {
|
||||
/// 使用多镜像配置创建容器
|
||||
pub async fn create_container_with_service_type(
|
||||
&self,
|
||||
mut config: DockerContainerConfig,
|
||||
service_type: ServiceType,
|
||||
project_overrides: Option<ProjectImageOverrides>,
|
||||
) -> DockerResult<DockerContainerInfo> {
|
||||
// 选择合适的镜像
|
||||
let image_selector = ImageSelector::new(self.get_multi_image_config());
|
||||
let selected_image = image_selector.select_image(&service_type, project_overrides.as_ref())?;
|
||||
|
||||
// 更新容器配置
|
||||
config.image = selected_image;
|
||||
|
||||
// 添加服务特定的环境变量
|
||||
if let Some(service_config) = image_selector.get_service_config(&service_type) {
|
||||
for (key, value) in &service_config.environment {
|
||||
config.env_vars.insert(key.clone(), value.clone());
|
||||
}
|
||||
|
||||
// 添加服务特定的挂载点
|
||||
for mount_config in &service_config.mounts {
|
||||
let host_path = self.resolve_host_path(&mount_config.host_path, &config.project_id)?;
|
||||
config.extra_mounts.push(ExtraMount {
|
||||
host_path,
|
||||
container_path: mount_config.container_path.clone(),
|
||||
read_only: mount_config.read_only,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// 使用现有逻辑创建容器
|
||||
self.create_container(config).await
|
||||
}
|
||||
|
||||
/// 获取多镜像配置
|
||||
fn get_multi_image_config(&self) -> MultiImageConfig {
|
||||
self.config.get_multi_image_config()
|
||||
}
|
||||
|
||||
/// 解析宿主机路径(支持变量替换)
|
||||
fn resolve_host_path(&self, path_template: &str, project_id: &str) -> DockerResult<String> {
|
||||
let resolved = path_template
|
||||
.replace("{project_id}", project_id)
|
||||
.replace("{workspace_dir}", &self.config.default_work_dir);
|
||||
|
||||
Ok(resolved)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. 项目级配置支持
|
||||
|
||||
支持在项目目录中创建 `.rcoder-image.yml` 文件来覆盖镜像配置:
|
||||
|
||||
```yaml
|
||||
# .rcoder-image.yml (项目级配置)
|
||||
project_id: "my-special-project"
|
||||
service_type: "agent-runner" # 可选择使用 AgentRunner 服务
|
||||
|
||||
# 项目特定的镜像覆盖
|
||||
images:
|
||||
rcoder: "my-custom-registry/rcoder:custom-v1.0"
|
||||
agent-runner: "my-custom-registry/agent-runner:custom-v1.0"
|
||||
|
||||
# 启用的服务类型
|
||||
enabled_services:
|
||||
- "rcoder"
|
||||
- "agent-runner"
|
||||
|
||||
# 项目特定的环境变量
|
||||
environment:
|
||||
AGENT_MODE: "production"
|
||||
LOG_LEVEL: "debug"
|
||||
RCODER_FEATURES: "full"
|
||||
```
|
||||
|
||||
### 4. API 接口扩展
|
||||
|
||||
扩展现有的聊天接口支持服务类型选择:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Deserialize, Serialize)]
|
||||
pub struct ChatRequest {
|
||||
pub prompt: String,
|
||||
pub project_id: Option<String>,
|
||||
pub session_id: Option<String>,
|
||||
pub attachments: Vec<Attachment>,
|
||||
|
||||
// 必填:服务类型选择 (强制要求指定)
|
||||
pub service_type: String, // "rcoder" 或 "agent-runner",不允许为空
|
||||
|
||||
// 可选:项目级镜像覆盖
|
||||
pub image_overrides: Option<HashMap<String, String>>,
|
||||
|
||||
// 现有字段...
|
||||
pub model_provider: Option<ModelProviderConfig>,
|
||||
pub request_id: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
## 📋 实施计划
|
||||
|
||||
### 阶段 1: 基础结构(1 天)
|
||||
1. 定义简化的 ServiceType (RCoder, AgentRunner)
|
||||
2. 实现基础的 MultiImageConfig
|
||||
3. 更新配置文件解析逻辑
|
||||
|
||||
### 阶段 2: 镜像选择器(1-2 天)
|
||||
1. 实现简化的 ImageSelector 核心逻辑
|
||||
2. 添加镜像缓存机制
|
||||
3. 实现向后兼容性支持
|
||||
|
||||
### 阶段 3: 容器创建集成(1-2 天)
|
||||
1. 更新 DockerManager 接口
|
||||
2. 集成服务类型到容器创建流程
|
||||
3. 实现项目级配置支持
|
||||
|
||||
### 阶段 4: API 和测试(1-2 天)
|
||||
1. 扩展 API 接口支持服务类型选择
|
||||
2. 更新文档和示例
|
||||
3. 编写核心测试用例
|
||||
|
||||
### 阶段 5: 部署和监控(1 天)
|
||||
1. 更新部署脚本和文档
|
||||
2. 添加基础监控和日志
|
||||
3. 简化性能测试
|
||||
|
||||
## 🔍 测试策略
|
||||
|
||||
### 单元测试
|
||||
- 镜像选择逻辑测试
|
||||
- 配置解析测试
|
||||
- 缓存机制测试
|
||||
|
||||
### 集成测试
|
||||
- 容器创建流程测试
|
||||
- 多服务类型协同测试
|
||||
- 项目级配置覆盖测试
|
||||
|
||||
### 端到端测试
|
||||
- 完整的聊天流程测试
|
||||
- 不同服务类型的容器启动测试
|
||||
- 配置热更新测试
|
||||
|
||||
## 📊 性能考虑
|
||||
|
||||
### 镜像缓存
|
||||
- 内存缓存减少重复计算
|
||||
- TTL 机制避免过期配置
|
||||
- LRU 策略控制内存使用
|
||||
|
||||
### 并发安全
|
||||
- DashMap 用于线程安全的缓存访问
|
||||
- 读写锁保护配置更新
|
||||
- 原子操作确保一致性
|
||||
|
||||
## 🔒 安全考虑
|
||||
|
||||
### 配置验证
|
||||
- 镜像名称格式验证
|
||||
- 路径注入防护
|
||||
- 环境变量安全检查
|
||||
|
||||
### 访问控制
|
||||
- 项目级配置权限控制
|
||||
- 服务类型启用/禁用控制
|
||||
- 镜像仓库访问权限
|
||||
|
||||
## 📈 监控和日志
|
||||
|
||||
### 指标收集
|
||||
- 镜像选择延迟
|
||||
- 缓存命中率
|
||||
- RCoder vs AgentRunner 使用统计
|
||||
|
||||
### 日志记录
|
||||
- 镜像选择决策日志
|
||||
- 配置解析错误日志
|
||||
- 容器创建失败日志
|
||||
|
||||
---
|
||||
|
||||
*本文档版本: v1.0*
|
||||
*最后更新: 2025-12-02*
|
||||
Reference in New Issue
Block a user