# Souin Response Truncation Bug Analysis

## Issue Summary
Large file downloads (e.g., 3MB tar.gz) are sometimes cached as truncated versions (~4KB) in Souin's OTTER backend.

**Behavior:**
- First request (cache miss): Downloads full 2.55MB ✓
- Second request (cache hit): Downloads only 3792 bytes (4KB) ✗

## Root Cause

The issue is a **race condition in Souin's response buffering during streaming**:

1. **Application Handler (pgs/web_asset_handler.go:304):**
   ```go
   w.WriteHeader(status)
   _, err := io.Copy(w, contents)  // Streaming large file to ResponseWriter
   ```

2. **Souin Middleware Flow (middleware.go:1085-1117):**
   ```go
   // Upstream() runs in a goroutine
   go func(vr *http.Request, cw *CustomWriter) {
       errorCacheCh <- s.Upstream(cw, vr, next, ...)  // <- blocks until complete
   }(req, customWriter)

   select {
   case <-req.Context().Done():
       // ...
   case v := <-errorCacheCh:
       _, _ = customWriter.Send()  // <- Called AFTER Upstream returns
   }
   ```

3. **Upstream Function (middleware.go:542-580):**
   ```go
   sfValue, err, shared := s.singleflightPool.Do(singleflightCacheKey, func() (interface{}, error) {
       if e := next(customWriter, rq); e != nil {  // <- Calls app handler
           // ...
       }
       // ...
       err := s.Store(customWriter, rq, ...)  // <- Caches response HERE
       defer customWriter.handleBuffer(func(b *bytes.Buffer) {
           b.Reset()  // <- Resets buffer after Store completes
       })
       return singleflightValue{
           body: customWriter.Buf.Bytes(),  // <- Captured BEFORE reset
           // ...
       }, err
   })
   ```

4. **Store Function (middleware.go:378-402):**
   ```go
   customWriter.mutex.Lock()
   b := customWriter.Buf.Bytes()  // <- Reads buffer contents
   bLen := customWriter.Buf.Len()
   customWriter.mutex.Unlock()

   res := http.Response{
       Body: io.NopCloser(bytes.NewBuffer(b)),  // <- Creates response with buffer snapshot
   }
   // ...
   response, err := httputil.DumpResponse(&res, true)  // <- Serializes for cache storage
   // Store response to cache
   ```

## The Problem

**Timing Issue with Streaming:**

When the application streams data via `io.Copy(w, contents)`, this happens **concurrently** with Souin's caching:

1. **Time T0:** `io.Copy()` starts streaming file data to CustomWriter
   - Data arrives in chunks via `Write()` calls
   - Souin buffers all data in `customWriter.Buf`

2. **Time T1:** First chunk (~4KB) is buffered
   - Souin's `Store()` is called (inside Upstream singleflight closure)
   - Reads `customWriter.Buf.Bytes()` at this moment
   - **If `io.Copy()` is still executing, only partial data is in the buffer!**

3. **Time T2:** The rest of the file continues streaming
   - But Store has already captured only 4KB
   - Cache stores truncated response

## Why It's Intermittent

- **Sometimes works:** If the entire file is buffered BEFORE Store is called
- **Sometimes fails:** If Store reads the buffer while io.Copy is mid-stream

The intermittency depends on:
- File size
- Network speed
- System load (goroutine scheduling)
- HTTP/2 vs HTTP/1.1 framing

## Solution Applied

**In pgs/web_asset_handler.go (L303-311):**

```go
w.WriteHeader(status)
_, err := io.Copy(w, contents)

if err != nil {
    logger.Error("io copy", "err", err.Error())
}

// Ensure all data is flushed to the client and cache before closing connection
// This prevents Souin's response buffer from being truncated mid-stream
if flusher, ok := w.(http.Flusher); ok {
    flusher.Flush()
}
```

**Why this helps:**
- `Flush()` forces HTTP/2 frame boundaries to be sent
- Tells Souin the response is complete before the handler returns
- Prevents Souin from caching partial responses

## Proper Long-Term Fix (in Souin)

Souin should implement one of:

1. **Wait for io.Copy() to complete before reading buffer:**
   - Don't call `Store()` until the handler function fully returns
   - Currently Store is called during Upstream while app handler may still be executing

2. **Use a completion signal:**
   - Application signals when response is complete (e.g., CloseNotifier)
   - Souin only snapshots buffer after this signal

3. **Buffer all data before storing:**
   - Ensure CustomWriter fully buffers before any Store operation
   - Requires synchronization changes

4. **Implement streaming cache validation:**
   - Store only cacheablecomplete responses
   - Validate Content-Length matches actual data before caching

## Files Affected

- `/home/erock/dev/pico/souin/pkg/middleware/writer.go` - CustomWriter
- `/home/erock/dev/pico/souin/pkg/middleware/middleware.go` - Upstream/Store logic
- `/home/erock/dev/pico/pkg/apps/pgs/web_asset_handler.go` - Application handler (FIXED)