Consistent Reverse Proxying in Go

Basic request proxying to a consistent address requires two things:

  1. Knowing where to route any given request.
  2. Being able to proxy the request itself.

Say we wanted to count the frequency of candy types given out at halloween, and because there is oh so much candy, we need to run many different servers in order to cope with the load. We decide we need to keep all counts for a particular candy type on a single machine - in order to effectively keep count without having to consult every other machine.

The first step is to be able to split the types of candy across different machines. To do this, and to get consistent results, we need to use consistent hashing. I like the stathat library for this. We can set it up as follows:

import "stathat.com/c/consistent"

addresses := []string{"https://myserver1.com:8765", "https://myserver2.com:8765", "https://myserver3.com:8765"}
routingHash := consistent.New()
routingHash.Set(addresses)

This sets up a routing hash where we have (default) 20 entries per server, to better split the load and make sure when adding or removing servers we don’t have to move all entries. Looking up a particular address is easy:

candyAddress, err := routingHash.Get("candy corn")
if err != nil {
    // something isn't right
}

Now we have our address, we need to proxy requests to a particular machine. We can setup a lookup hash:

import "net/http/httputil"

proxyMap := map[string]*httputil.ReverseProxy{}
for _, addr := range addresses {
    proxyMap[addr] = httputil.NewSingleHostReverseProxy(addr)
}

Here we use the standard library reverse proxy, and just map a machine address to this proxy. We can do this in a HTTP handler by reading the body to get the candy name to route to, and then using our consistent hash plus proxy map.

type CandyRequest struct {
  Name string
}

func ProxyTo(w http.ResponseWriter, r *http.Request) {
    // make two copies, one we write back
    buf, _ := ioutil.ReadAll(r.Body)
    readerCopy := ioutil.NopCloser(bytes.NewBuffer(buf))
    writeBackCopy := ioutil.NopCloser(bytes.NewBuffer(buf))
    r.Body = writeBackCopy

    // decode request
    request := CandyRequest{}
    decoder := json.NewDecoder(readerCopy)
  	decoder.Decode(&request)

    // use our routing hash and proxy map
    candyAddress, _ := getRoutingHash().Get(request.Name)
    proxy := getProxyMap()[candyAddress]
    proxy.ServeHTTP(w, r)
}

There’s a bunch of improvements that could be made here; instead of copying the body back and forth we could include the routing information as a header, and adding ways to deal with servers being down with routing to secondaries. Also notably absent, no coordination between nodes - though that may be an OK tradeoff.