Globally available NGINX proxy using Kubernetes federation

Even though each Open Targets release is made globally available in 3 regions, we decided early on to mask this complexity from the user and publish our API from a single URL at https://api.opentargets.io

We achieve this by load balancing our traffic from https://api.opentargets.io to three reverse proxies in three different regions. Here is a tutorial on how to recreate the same setup, using kubernetes on Google Cloud Platform.

Motivation

Publishing our platform to a global audience, we have always strived to maintain a fast and responsive user experience. How fast our webapp is, directly depends on how fast our API is. In particular, how fast it can responds to each query from the webapp. This is measured by latency, which is the average time that passes from the moment our webapp requests the data to the time it receives it). Latencies are a product of i) how long it takes to compute the data and ii) how far it has to travel. The farther a signal has to travel, the longer it takes and the larger the latency.

After optimizing how long it takes to compute the data, the most effective way to achieve low latencies is therefore to have a large
number of servers, distributed around the world that serve our data. However, lots of
servers mean lots of work in keeping them up, synchronized and easy to maintain, which can be quite hard for a team as small as ours.

While we found other great ready-made solutions to deploy our webapp,
we had to build our solution for our API and backend. In an effort to keep operational work to a minimum we choose to use performant cloud solutions, specifically Google Cloud compute and Google App Engine.

Tutorial

We will address how we automatically deploy our data and API in three regions, using continuos integration in another blog post.

Here we are going to describe how we tie it all together and mask this complexity from the user, while exposing a single https://api.opentargets.io URL.

We wanted to load balance and allocate the traffic from https://api.opentargets.io to three reverse proxies in three different regions (one in the United States, one in Europe and one in Asia), which in turn directed the traffic to our backends replicas - each running in a separate regional AppEngine project.

The choice of proxy was obvious to us: we have always been a fan of nginx and used it as a reverse proxy in a variety of scenario.
We choose to orchestrate the nginx instances using a kubernetes federated cluster, which will allow us to manipulate the configuration of nginx on a geographically distributed way, and with minimal ops work.

Prerequisites

You need gcloud and kubectl command-line tools installed and set up to run deployment commands.

For production, you also need to install kubefed.

Warning! you need to download the latest version yourself. It doesn't come with the google-cloud-sdk, but you will think it does and thus assume its version is in sync with the generally available Kubernetes version.
Don't make your life miserable. Always install the latest version directly.

1. Make a DNS zone

I created the apiproxy.opentargets.io zone on google cloud DNS

2. Deploy a GKE cluster

Make the clusters, one node each. The n1-standard-1 machine is the default. They all need to have google CloudDNS roles attached to the instance, so you must create them from the command line.

gcloud --project open-targets-prod container clusters create apiproxy-eu \  
  --zone=europe-west1-d --quiet \
    --num-nodes=1 \
  --scopes cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite \
  > /dev/null 2>&1 &

gcloud --project open-targets-prod container clusters create apiproxy-useast \  
  --zone=us-east1-d --quiet \
  --num-nodes=1 \
  --scopes cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite

gcloud --project open-targets-prod container clusters create apiproxy-jp \  
  --zone=asia-northeast1-b --quiet \
  --num-nodes=1 \
  --scopes cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite

You have to force use of credentials certificate, otherwise it will use the temporary user token of your kubectl contexts and render the federation unoperable after a few minutes.

gcloud --project open-targets-prod config set container/use_client_certificate True  
export CLOUDSDK_CONTAINER_USE_CLIENT_CERTIFICATE=True  

Then you can type the suggested commands:

gcloud container clusters get-credentials apiproxy-useast --zone us-east1-d --project open-targets-prod

gcloud container clusters get-credentials apiproxy-jp --zone asia-northeast1-b --project open-targets-prod

gcloud container clusters get-credentials apiproxy-eu --zone europe-west1-d --project open-targets-prod  

3. Federate the clusters

Initiate the cluster federation by launching:

kubefed init apiproxyfed --host-cluster-context=gke_open-targets-prod_europe-west1-d_apiproxy-eu \  
     --dns-provider="google-clouddns" \
     --dns-zone-name="apiproxy.opentargets.io.

you also have to create a namespace, since kubefed does not do this for you.

kubectl create ns default ##<= wrong, check the right syntax  

From here onwards we assume that kubectl uses the apiproxyfed if not otherwise specified.

Rename clusters context to be DNS compatible (no long words with underscores)

kubectl config rename-context gke_open-targets-prod_europe-west1-d_apiproxy-eu apiproxy-eu

kubectl config rename-context gke_open-targets-prod_asia-northeast1-b_apiproxy-jp apiproxy-jp

kubectl config rename-context gke_open-targets-prod_us-east1-d_apiproxy-useast apiproxy-useast  

join the renamed clusters to the federation plane

kubefed join us-apiproxy --cluster-context="apiproxy-useast" --host-cluster-context="apiproxy-eu"

kubefed join eu-apiproxy --cluster-context="apiproxy-eu" --host-cluster-context="apiproxy-eu"

kubefed join jp-apiproxy --cluster-context="apiproxy-jp" --host-cluster-context="apiproxy-eu  

To test that it's all working, you can try spin 3 nginx replicas across the 3 regions. Notice that deployment is currently under extensions/v1beta1.

kubectl create deployment nginx --image=nginx && kubectl scale deployment nginx --replicas=3  

You should see three deployments, one replicas each, in each of the clusters. Check with:

kubectl --context=apiproxy-eu get svc,po,deploy  

Should you use another namespace?

If you want to try spinning another version of the proxy (eg. if you want to spin your proxy servers with a new nginx.conf, but keep production up), this is a good time to create another namespace and repeat all the steps below.

Use your server.key and server.chained.crt files for your domain:

kubectl --context apiproxyfed create secret tls opentargetsio-certs --key server.key --cert server.chained.crt  

4. Deploy your API proxy

We can deploy the config map and the services to the federated cluster, but the deployments will be region specific (so that each points to the right service).

kubectl config set-context apiproxy-fed  

Create the default nginx configmap, and the location specific one:

kubectl create configmap nginxconf --from-file=nginx.conf

kubectl create configmap backends --from-file=eu=backend.eu.conf --from-file=us=backend.us.conf --from-file=jp=backend.jp.conf  

Take a look at it:

kubectl describe configmap backends  

We are spinning a nodeport and a loadbalancer, because a loadbalancer would give us three separate IPs. First we reserve the IP:

# create the IP
gcloud --project open-targets-prod compute addresses create api-ingress --global  

Then spin the nginx deployment,service and ingress:

kubectl create -f apiproxy_deploy.yaml  
kubectl create -f apiproxy_svc.yaml  
kubectl create -f apiproxy_ing.yaml  

where respectively:

apiVersion: extensions/v1beta1  
kind: Deployment  
metadata:  
  name: apiproxy-eu
spec:  
  replicas: 1
  template:
    metadata:
      labels:
        name: apiproxy-eu
        app: apiproxy
        region: eu
    spec:
      volumes:
        - name: nginxconf-volume
          configMap:
            name: nginxconf
        - name: backends-volume
          configMap:
            name: backends
            items:
            - key: proxy
              path: proxy.conf
            - key: eu
              path: backend.conf
      containers:
      - image: nginx:1.13-alpine
        name: nginx
        livenessProbe:
          httpGet:
            path: /_heartbeat
            port: 80
          initialDelaySeconds: 30
          timeoutSeconds: 1
        readinessProbe:
          httpGet:
            path: /_ah/health
            port: 80
            httpHeaders:
              - name: Host
                value: api.opentargets.io
          initialDelaySeconds: 30
          timeoutSeconds: 10
        volumeMounts:
          - mountPath: /etc/nginx/nginx.conf
            name: nginxconf-volume
            subPath: nginx.conf
          - mountPath: /etc/nginx/conf.d/
            name: backends-volume
        ports:
        - name: http
          containerPort: 80
          protocol: TCP
apiVersion: v1  
kind: Service  
metadata:  
  name: apiproxysvc-node
  labels:
    name: apiproxysvc-node
spec:  
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
    nodePort: 30036
  selector:
    app: apiproxy
apiVersion: extensions/v1beta1  
kind: Ingress  
metadata:  
  name: apiproxying-tls
  annotations:
    # kubernetes.io/tls-acme: "true"
    kubernetes.io/ingress.class: "gce"
    kubernetes.io/ingress.global-static-ip-name: “api-ingress”
spec:  
  backend:
    serviceName: apiproxysvc-node
    servicePort: 80
  tls:
  - secretName: opentargetsio-certs

NOTE this can take a long time before it's working and all the "backends" look green and healthy in the GKE dashboard. Go fix yourself a coffee.

You can monitor the progress directly from the console.

Final steps

You should verify that the proxy listens (preferrably from different geographical locations):

curl http://YOURAPI.URL/  

and when you are happy with its functionality, you can also enable CDN, with a command similar to:

BACKEND=$(kubectl get ing apiproxy -o json | jq -j '.metadata.annotations."ingress.kubernetes.io/backends"' | jq -j 'keys[0]')  
gcloud --project open-targets-prod compute backend-services update $BACKEND --enable-cdn  

Hope this tutorial helps you testing out the kubernetes federation functionality! Let me know in the comments if you have trouble.