TCP Connections vs HTTP Requests: Why a Byte-Copying Proxy Works Until You Need Per-Request Anything

I've configured nginx, Traefik, and HAProxy more times than I can count. Point it at some backends, set a timeout, move on. What I'd never actually done was build one, so I started RelayForge, a reverse proxy and load balancer in Go, from the absolute simplest version possible: a program that accepts a connection, opens one to a backend, and copies bytes between them.

That version works. Not "works for a toy case", it genuinely forwards real HTTP traffic correctly, persistent connections and all. Understanding exactly why it works, and exactly where it stops being enough, turned out to be the most useful thing about building it first.

The simplest proxy that could possibly work

func (s *Server) handleConn(client net.Conn) {
    defer client.Close()
    backend, err := net.Dial("tcp", s.Backend)
    if err != nil {
        return
    }
    defer backend.Close()

    done := make(chan struct{}, 2)
    go func() {
        io.Copy(backend, client)
        done <- struct{}{}
    }()
    go func() {
        io.Copy(client, backend)
        done <- struct{}{}
    }()
    <-done
    <-done
}

That's genuinely most of it. Accept a connection, dial the one configured backend, copy bytes in both directions until either side hangs up. No parsing, no awareness that HTTP is even involved. And if you point a real browser or curl at it, it works, because TCP doesn't know or care what's inside the stream. A client can open a persistent HTTP/1.1 connection, send several requests down it, and this proxy will happily shuttle every byte of all of them back and forth without ever noticing there was more than one request.

So why isn't this the whole project?

Because "it forwards bytes correctly" and "it can make a decision about one request" are completely different capabilities, and the byte-copying version only has the first one.

Say you want to load-balance across more than one backend. With a byte-copying proxy, the earliest point you could possibly make that decision is when the client connection first opens, because that's the only moment you have a backend connection to dial at all. Once it's open, it's open, every byte that flows through it for the rest of that TCP connection's life goes to the same backend. If a client keeps one persistent connection open for an hour and sends a thousand requests down it, all thousand go to whichever backend got picked at the start. That's not load balancing, that's one really sticky decision.

Retries have the same problem, worse. If a backend fails partway through, what exactly would you retry? You don't have "the request" as a thing you could resend elsewhere, you have an open socket with some bytes already flowed through it in one direction and who knows what in the other. There's no boundary to even talk about retrying a request, because nothing in this code has ever identified where one request ends and the next begins.

Metrics, pooling, cancellation tied to a specific client giving up mid-request, they all have the same shape of problem. Every one of them needs to reason about a request, and a byte-copying proxy has no concept of a request at all. It has a stream.

Giving the proxy a request boundary

The fix is to stop copying bytes blindly and start parsing HTTP:

func (s *Server) forwardOneRequest(client net.Conn, clientReader *bufio.Reader) bool {
    req, err := http.ReadRequest(clientReader)
    if err != nil {
        return false
    }
    // ...forward req to a backend, read the matching response...
    return !req.Close && !resp.Close
}

http.ReadRequest (and http.ReadResponse on the way back) know how to find where a message actually ends, using Content-Length when it's present, chunked transfer-encoding when it isn't, and the connection-close framing rules for the cases where neither tells you directly. That's the real content of "HTTP awareness": not printing a method and a path, but knowing precisely where one message's bytes stop and the next one's begin, in a stream where TCP itself gives you no such marker.

Once you have that, a persistent connection goes from "one long undifferentiated stream" to "a sequence of distinct messages, each with its own boundary":

func (s *Server) handleConn(client net.Conn) {
    defer client.Close()
    clientReader := bufio.NewReader(client)
    for s.forwardOneRequest(client, clientReader) {
    }
}

Same persistent connection as before, same client behavior. But now the loop calls http.ReadRequest fresh for every request, which means every single one of them is a point where the proxy can make a brand new decision: which backend, whether to retry, whether to count it in a metric. I proved this to myself directly in testing: round-robining two backends across six requests sent down one persistent client connection landed exactly backend-0, backend-1, backend-2, backend-0, backend-1, backend-2, alternating every single request despite the TCP connection underneath never closing once. A byte-copying proxy could never produce that pattern, there'd be nothing in it capable of noticing a second request had even started.

The part that's easy to get backwards

It's tempting to think HTTP parsing is what makes a reverse proxy correct, and the byte-copying version was some kind of broken first draft. It wasn't broken, it correctly forwarded everything TCP alone is responsible for. What it couldn't do was anything that requires acting on a request as its own unit, which is a different, additional thing you only need once you want per-request routing, retries, or metrics. Milestone by milestone, in RelayForge, everything after this one (load balancing, health checks, connection pooling, retries) is built on exactly that boundary. None of it would have anywhere to attach without it.