1# Proposal: Transparent Rootless TCP Record & Replay for `pici`
2
3**Status**: Draft
4**Target Platform**: Linux (x86_64, aarch64)
5**Dependencies**: Zero external runtime dependencies (Pure Go binary)
6
7---
8
9## 1. Executive Summary & Objective
10
11When iterating on local CI pipelines or writing test suites against `pici`, running real tasks often triggers destructive or stateful side effects—such as uploading GitHub releases, pushing Git tags, deploying to SSH servers, or consuming rate-limited third-party APIs.
12
13This proposal introduces a **fully transparent, zero-configuration TCP Record & Replay engine** built natively into `pici`. By leveraging rootless Linux kernel namespaces (`CLONE_NEWUSER`, `CLONE_NEWNET`, `CLONE_NEWNS`) and an in-process user-space TCP/IP stack (`gVisor netstack`), `pici` can intercept, record, and mock all outbound network traffic (HTTP, HTTPS, SSH, Git) without requiring `sudo`, environment variable modifications, or tool-specific configuration.
14
15---
16
17## 2. System Architecture
18
19```mermaid
20flowchart TB
21 subgraph Host["pici Host Process (Go Runtime)"]
22 NETSTACK["In-Process TCP/IP Stack\n(gVisor pkg/tcpip)"]
23 RECORDER["Record / Replay Engine\n(L4 Stream + L7 HTTP/SSH Parsers)"]
24 CASSETTE[("Cassette Store\n.pici/cassettes/<name>/")]
25
26 NETSTACK <--> RECORDER <--> CASSETTE
27 end
28
29 subgraph Sandbox["Rootless Sandbox (unshare -U -n -m)"]
30 TUN["Virtual TUN Device (Default Route: 10.0.2.1/24)"]
31 SSL_MOUNT["Ephemeral /etc/ssl Trust Store\n(Dynamically mounted in-memory)"]
32
33 subgraph Tasks["pico.sh & Subprocesses (zmx / Docker / CLI tools)"]
34 T1["curl / npm / apt / gh"]
35 T2["git push git@github.com:..."]
36 T3["ssh pipe.pico.sh 'pub build.status'"]
37 end
38 end
39
40 Tasks -->|"Raw Outbound TCP (All Ports & IPs)"| TUN
41 TUN -->|"Packet Stream (UNIX Socketpair FD)"| NETSTACK
42 RECORDER -.->|"Record Mode: Real WAN\nReplay Mode: Serves Cassettes"| WAN["External Network / APIs"]
43```
44
45---
46
47## 3. Kernel Primitives & Sandbox Isolation
48
49The sandbox is spawned directly by `pici` without external utilities (`slirp4netns` or `podman` are not required):
50
51### 1. User & Network Namespace (`CLONE_NEWUSER | CLONE_NEWNET`)
52* `pici` spawns the job wrapper with `CLONE_NEWUSER` and sets a single UID/GID mapping (`Host UID -> Container UID 0`). This gives the child process full administrative privileges *inside the namespace only*, with zero privileges on the host.
53* `CLONE_NEWNET` provides a completely isolated network stack with its own loopback and routing tables.
54* A virtual `TUN` device is created and set as the default gateway (`0.0.0.0/0 -> 10.0.2.1`).
55
56### 2. Mount Namespace (`CLONE_NEWNS`) for Ephemeral TLS Trust
57* To intercept HTTPS traffic without modifying the user's host system, `pici` generates an in-memory Root CA keypair.
58* Inside the mount namespace, `pici` bind-mounts a generated bundle containing this CA over standard CA trust paths:
59 * `/etc/ssl/certs/ca-certificates.crt`
60 * `/etc/pki/tls/certs/ca-bundle.crt`
61 * `/etc/ssl/cert.pem`
62* When the job finishes and the namespace is destroyed, all certificate mounts disappear without leaving any trace on the host.
63
64---
65
66## 4. Transparent Protocol Handlers
67
68### 1. HTTP & HTTPS
69* **TLS Termination**: When the user-space stack receives a connection to port 443 (or any TLS port), `pici` dynamically mints a TLS certificate for the target SNI hostname signed by the ephemeral Root CA.
70* **Recording**: Captures `{ method, url, request_headers, request_body }` and paired `{ status_code, response_headers, response_body }`.
71* **Header Normalization**: Headers like `Date`, `User-Agent`, `X-Request-Id`, and authentication tokens are filtered or normalized to ensure deterministic replay matching.
72
73### 2. SSH & Git over SSH (`git push`, `ssh pipe.pico.sh`)
74* When traffic to port 22 or starting with the `SSH-2.0-` banner is intercepted:
75 * **Record Mode**: `pici` performs a transparent SSH proxy handshake, records the server host key, captures channel requests (`exec`, `subsystem`, `pty`), and streams data bidirectionally while logging the conversation.
76 * **Replay Mode**: `pici` completes the SSH handshake using the recorded host key, accepts public key or password auth unconditionally, and replays recorded stdout/stderr streams and exit codes for matching commands.
77
78### 3. Raw TCP Fallback
79* For unrecognized protocols or raw TCP sockets, `pici` records sequential bidirectional packet streams indexed by `Destination IP:Port`.
80
81---
82
83## 5. Cassette Storage & Secret Sanitization
84
85Cassettes are stored in `.pici/cassettes/<name>/` as clean, human-readable JSON files designed to be checked into source control:
86
87```
88.pici/cassettes/release-v1.0/
89├── manifest.json # Metadata, timestamps, sanitized environment
90├── http.jsonl # HTTP/HTTPS requests & responses
91└── ssh.jsonl # SSH exec commands, terminal streams, exit statuses
92```
93
94### Automatic Secret Scrubbing
95Before writing any cassette to disk:
961. `pici` inspects all active environment variables (e.g. `GITHUB_TOKEN`, `AWS_SECRET_ACCESS_KEY`, `SSH_AUTH_SOCK`).
972. Any string matching an active secret value in headers or payloads is replaced with a deterministic placeholder: `{{ REDACTED_SECRET_<KEY> }}`.
983. During **Replay Mode**, the placeholder is dynamically matched against whatever secret is passed in the local environment.
99
100---
101
102## 6. Developer Experience & CLI Interface
103
104### CLI Commands
105```bash
106# 1. Execute job live and record all network traffic to default cassette (.pici/cassettes/default)
107pici --record
108
109# 2. Execute job live and record to a named cassette
110pici --record-to .pici/cassettes/release-flow
111
112# 3. Replay job completely offline (0 network calls, 0 side effects)
113pici --replay-from .pici/cassettes/release-flow
114
115# 4. Strict Replay Mode (fails immediately on any unrecorded network call)
116pici --replay-from .pici/cassettes/release-flow --strict-network
117```
118
119### Go Test Harness Integration
120Developers can write end-to-end tests for their CI pipeline directly in Go:
121
122```go
123func TestPicoCI_ReleaseJob(t *testing.T) {
124 runner := pici.NewRunner(&pici.Config{
125 WorkspaceDir: ".",
126 ReplayCassette: "./testdata/cassettes/release-flow",
127 StrictNetwork: true,
128 })
129
130 res, err := runner.Run(context.Background())
131 require.NoError(t, err)
132 require.Equal(t, 0, res.ExitCode)
133}
134```
135
136---
137
138## 7. Implementation Roadmap
139
140| Milestone | Deliverables |
141| :--- | :--- |
142| **Phase 1: Sandbox & TCP Bridge** | Implement `unshare` runner with `CLONE_NEWUSER \| CLONE_NEWNET \| CLONE_NEWNS`, create in-memory TUN socketpair, and wire into `gvisor.dev/gvisor/pkg/tcpip`. |
143| **Phase 2: HTTP/S MITM & TLS Mounting** | In-memory CA generation, ephemeral `/etc/ssl` bind mounting, dynamic SNI cert generation, and JSONL HTTP cassette recorder/replayer. |
144| **Phase 3: SSH & Git MITM Engine** | Transparent SSH proxy handler for `git push` and `ssh ...` commands; recording exec channel payloads and exit codes. |
145| **Phase 4: Secret Sanitization & CLI Flags** | Automatic token redaction, `--record` and `--replay` CLI flags, summary status output in UI reporter. |
146
147---
148
149> [!NOTE]
150> This design maintains `pici`'s zero-dependency single-binary philosophy (`CGO_ENABLED=0 go build`) and works seamlessly on any modern Linux kernel without requiring root or modifying host configurations.