# Proposal: Transparent Rootless TCP Record & Replay for `pici`

**Status**: Draft  
**Target Platform**: Linux (x86_64, aarch64)  
**Dependencies**: Zero external runtime dependencies (Pure Go binary)  

---

## 1. Executive Summary & Objective

When iterating on local CI pipelines or writing test suites against `pici`, running real tasks often triggers destructive or stateful side effects—such as uploading GitHub releases, pushing Git tags, deploying to SSH servers, or consuming rate-limited third-party APIs.

This proposal introduces a **fully transparent, zero-configuration TCP Record & Replay engine** built natively into `pici`. By leveraging rootless Linux kernel namespaces (`CLONE_NEWUSER`, `CLONE_NEWNET`, `CLONE_NEWNS`) and an in-process user-space TCP/IP stack (`gVisor netstack`), `pici` can intercept, record, and mock all outbound network traffic (HTTP, HTTPS, SSH, Git) without requiring `sudo`, environment variable modifications, or tool-specific configuration.

---

## 2. System Architecture

```mermaid
flowchart TB
    subgraph Host["pici Host Process (Go Runtime)"]
        NETSTACK["In-Process TCP/IP Stack\n(gVisor pkg/tcpip)"]
        RECORDER["Record / Replay Engine\n(L4 Stream + L7 HTTP/SSH Parsers)"]
        CASSETTE[("Cassette Store\n.pici/cassettes/<name>/")]
        
        NETSTACK <--> RECORDER <--> CASSETTE
    end

    subgraph Sandbox["Rootless Sandbox (unshare -U -n -m)"]
        TUN["Virtual TUN Device (Default Route: 10.0.2.1/24)"]
        SSL_MOUNT["Ephemeral /etc/ssl Trust Store\n(Dynamically mounted in-memory)"]

        subgraph Tasks["pico.sh & Subprocesses (zmx / Docker / CLI tools)"]
            T1["curl / npm / apt / gh"]
            T2["git push git@github.com:..."]
            T3["ssh pipe.pico.sh 'pub build.status'"]
        end
    end

    Tasks -->|"Raw Outbound TCP (All Ports & IPs)"| TUN
    TUN -->|"Packet Stream (UNIX Socketpair FD)"| NETSTACK
    RECORDER -.->|"Record Mode: Real WAN\nReplay Mode: Serves Cassettes"| WAN["External Network / APIs"]
```

---

## 3. Kernel Primitives & Sandbox Isolation

The sandbox is spawned directly by `pici` without external utilities (`slirp4netns` or `podman` are not required):

### 1. User & Network Namespace (`CLONE_NEWUSER | CLONE_NEWNET`)
* `pici` spawns the job wrapper with `CLONE_NEWUSER` and sets a single UID/GID mapping (`Host UID -> Container UID 0`). This gives the child process full administrative privileges *inside the namespace only*, with zero privileges on the host.
* `CLONE_NEWNET` provides a completely isolated network stack with its own loopback and routing tables.
* A virtual `TUN` device is created and set as the default gateway (`0.0.0.0/0 -> 10.0.2.1`).

### 2. Mount Namespace (`CLONE_NEWNS`) for Ephemeral TLS Trust
* To intercept HTTPS traffic without modifying the user's host system, `pici` generates an in-memory Root CA keypair.
* Inside the mount namespace, `pici` bind-mounts a generated bundle containing this CA over standard CA trust paths:
  * `/etc/ssl/certs/ca-certificates.crt`
  * `/etc/pki/tls/certs/ca-bundle.crt`
  * `/etc/ssl/cert.pem`
* When the job finishes and the namespace is destroyed, all certificate mounts disappear without leaving any trace on the host.

---

## 4. Transparent Protocol Handlers

### 1. HTTP & HTTPS
* **TLS Termination**: When the user-space stack receives a connection to port 443 (or any TLS port), `pici` dynamically mints a TLS certificate for the target SNI hostname signed by the ephemeral Root CA.
* **Recording**: Captures `{ method, url, request_headers, request_body }` and paired `{ status_code, response_headers, response_body }`.
* **Header Normalization**: Headers like `Date`, `User-Agent`, `X-Request-Id`, and authentication tokens are filtered or normalized to ensure deterministic replay matching.

### 2. SSH & Git over SSH (`git push`, `ssh pipe.pico.sh`)
* When traffic to port 22 or starting with the `SSH-2.0-` banner is intercepted:
  * **Record Mode**: `pici` performs a transparent SSH proxy handshake, records the server host key, captures channel requests (`exec`, `subsystem`, `pty`), and streams data bidirectionally while logging the conversation.
  * **Replay Mode**: `pici` completes the SSH handshake using the recorded host key, accepts public key or password auth unconditionally, and replays recorded stdout/stderr streams and exit codes for matching commands.

### 3. Raw TCP Fallback
* For unrecognized protocols or raw TCP sockets, `pici` records sequential bidirectional packet streams indexed by `Destination IP:Port`.

---

## 5. Cassette Storage & Secret Sanitization

Cassettes are stored in `.pici/cassettes/<name>/` as clean, human-readable JSON files designed to be checked into source control:

```
.pici/cassettes/release-v1.0/
├── manifest.json       # Metadata, timestamps, sanitized environment
├── http.jsonl          # HTTP/HTTPS requests & responses
└── ssh.jsonl           # SSH exec commands, terminal streams, exit statuses
```

### Automatic Secret Scrubbing
Before writing any cassette to disk:
1. `pici` inspects all active environment variables (e.g. `GITHUB_TOKEN`, `AWS_SECRET_ACCESS_KEY`, `SSH_AUTH_SOCK`).
2. Any string matching an active secret value in headers or payloads is replaced with a deterministic placeholder: `{{ REDACTED_SECRET_<KEY> }}`.
3. During **Replay Mode**, the placeholder is dynamically matched against whatever secret is passed in the local environment.

---

## 6. Developer Experience & CLI Interface

### CLI Commands
```bash
# 1. Execute job live and record all network traffic to default cassette (.pici/cassettes/default)
pici --record

# 2. Execute job live and record to a named cassette
pici --record-to .pici/cassettes/release-flow

# 3. Replay job completely offline (0 network calls, 0 side effects)
pici --replay-from .pici/cassettes/release-flow

# 4. Strict Replay Mode (fails immediately on any unrecorded network call)
pici --replay-from .pici/cassettes/release-flow --strict-network
```

### Go Test Harness Integration
Developers can write end-to-end tests for their CI pipeline directly in Go:

```go
func TestPicoCI_ReleaseJob(t *testing.T) {
    runner := pici.NewRunner(&pici.Config{
        WorkspaceDir:   ".",
        ReplayCassette: "./testdata/cassettes/release-flow",
        StrictNetwork:  true,
    })
    
    res, err := runner.Run(context.Background())
    require.NoError(t, err)
    require.Equal(t, 0, res.ExitCode)
}
```

---

## 7. Implementation Roadmap

| Milestone | Deliverables |
| :--- | :--- |
| **Phase 1: Sandbox & TCP Bridge** | Implement `unshare` runner with `CLONE_NEWUSER \| CLONE_NEWNET \| CLONE_NEWNS`, create in-memory TUN socketpair, and wire into `gvisor.dev/gvisor/pkg/tcpip`. |
| **Phase 2: HTTP/S MITM & TLS Mounting** | In-memory CA generation, ephemeral `/etc/ssl` bind mounting, dynamic SNI cert generation, and JSONL HTTP cassette recorder/replayer. |
| **Phase 3: SSH & Git MITM Engine** | Transparent SSH proxy handler for `git push` and `ssh ...` commands; recording exec channel payloads and exit codes. |
| **Phase 4: Secret Sanitization & CLI Flags** | Automatic token redaction, `--record` and `--replay` CLI flags, summary status output in UI reporter. |

---

> [!NOTE]
> This design maintains `pici`'s zero-dependency single-binary philosophy (`CGO_ENABLED=0 go build`) and works seamlessly on any modern Linux kernel without requiring root or modifying host configurations.