Deploy a Golang App to Interact with the Kubernetes API

I was curious on how a application running on kubernetes can get access to kubernetes api's so that the application can get access to retrieve information about pods in a specific namespace.

This will also show us how to grant applications to specific resources using service accounts.

What will we be doing

In this tutorial we will do the following:

Install Go 1.20
Build a Go API that will check the pods on the Kubernetes cluster
Create RBAC to grant our application access to Kubernetes

Install Go

Install dependencies such as git, gcc and wget then install Go:

wget https://go.dev/dl/go1.22.3.linux-amd64.tar.gz
tar -xvf go1.20.3.linux-amd64.tar.gz
mv go /usr/local/go-1.22

Configure Go:

export GO111MODULE=on
export GOROOT=/usr/local/go-1.22
export GOPATH=~/go
export PATH=$GOROOT/bin:$GOPATH/bin:$PATH

You should be able to do:

go env GOROOT
go env GOPATH
go version

Create the Go application

Inside my ~/workspace/application we need to initialize our module:

go mod init healthcheck-api

Then in our main.go file we populate it with the following code:

main.go

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"os"

	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/rest"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

func main() {
	http.HandleFunc("/healthcheck/pods", listPodsHandler)
	http.HandleFunc("/healthcheck/pods/status", statusCodeHandler)

	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
	}

	log.Printf("Listening on port %s", port)
	if err := http.ListenAndServe(fmt.Sprintf(":%s", port), nil); err != nil {
		log.Fatalf("could not start server: %v", err)
	}
}

func listPodsHandler(w http.ResponseWriter, r *http.Request) {
	clientset, err := getK8sClient()
	if err != nil {
		http.Error(w, fmt.Sprintf("could not create clientset: %v", err), http.StatusInternalServerError)
		return
	}

	pods, err := clientset.CoreV1().Pods("default").List(context.TODO(), metav1.ListOptions{})
	if err != nil {
		http.Error(w, fmt.Sprintf("could not list pods: %v", err), http.StatusInternalServerError)
		return
	}

	w.Header().Set("Content-Type", "application/json")
	if err := json.NewEncoder(w).Encode(pods.Items); err != nil {
		http.Error(w, fmt.Sprintf("could not encode response: %v", err), http.StatusInternalServerError)
	}
}

func statusCodeHandler(w http.ResponseWriter, r *http.Request) {
	clientset, err := getK8sClient()
	if err != nil {
		http.Error(w, fmt.Sprintf("could not create clientset: %v", err), http.StatusInternalServerError)
		return
	}

	// Attempt to list the pods to check if the service is working
	_, err = clientset.CoreV1().Pods("default").List(context.TODO(), metav1.ListOptions{})
	if err != nil {
		http.Error(w, fmt.Sprintf("could not list pods: %v", err), http.StatusInternalServerError)
		return
	}

	// Set the response header to indicate JSON content
	w.Header().Set("Content-Type", "application/json")
	// Set the status code to 200 OK
	w.WriteHeader(http.StatusOK)

	// Create the response map
	response := map[string]int{"status": http.StatusOK}

	// Encode and write the response to the ResponseWriter
	if err := json.NewEncoder(w).Encode(response); err != nil {
		http.Error(w, fmt.Sprintf("could not encode response: %v", err), http.StatusInternalServerError)
	}
}


func getK8sClient() (*kubernetes.Clientset, error) {
	config, err := rest.InClusterConfig()
	if err != nil {
		return nil, fmt.Errorf("could not get in-cluster config: %v", err)
	}

	clientset, err := kubernetes.NewForConfig(config)
	if err != nil {
		return nil, fmt.Errorf("could not create clientset: %v", err)
	}

	return clientset, nil
}

In short, we have defined two functions:

listPodsHandler: this retrieves pods information from the default namespace.
statusCodeHandler: this is similar to above, but returns the http status code.

We then assign two routes to those functions:

/healthcheck/pods -> listPodsHandler
/healthcheck/pods/status -> statusCodeHandler

The application runs on port 8080.

Once you have the code saved, we can then run the following to add any missing modules using:

go mod tidy

Docker

Since we will be running this on Kubernetes, we have our Dockerfile:

Dockerfile

# Base image
FROM golang:1.22 AS builder

WORKDIR /app
COPY . .
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -installsuffix cgo -o healthcheck-api .

# Use a minimal base image
FROM gcr.io/distroless/base-debian10
WORKDIR /
COPY --from=builder /app/healthcheck-api /app/healthcheck-api

CMD ["/app/healthcheck-api"]

Test if the docker build works:

docker build -t go-healthcheck -f Dockerfile .

Now you can push it to your registry of choice.

Kubernetes

I will assume that you already have a Kubernetes cluster running, if not, you can check out this post to run a local Kubernetes cluster with kind:

https://ruan.dev/blog/2022/09/20/kind-for-local-kubernetes-clusters

Now we need to create the following RBAC resources on Kubernetes:

ServiceAccount
ClusterRole
ClusterRoleBinding

Create the following directory:

mkdir -p rbac

Then first we will create the rbac/serviceaccount.yaml:

rbac/serviceaccount.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-reader
  namespace: default

Then we proceed with rbac/clusterrole.yaml:

rbac/clusterrole.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]

Then lastly our rbac/clusterrolebinding.yaml:

rbac/clusterrolebinding.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: pod-reader-binding
subjects:
- kind: ServiceAccount
  name: pod-reader
  namespace: default
roleRef:
  kind: ClusterRole
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

In short, we are defining a service account, then we define a cluster role with permissions to list and get pods, then we define the cluster role binding.

You can apply that to the cluster to create the resources using:

kubectl apply -f rbac/

Now the only thing let to do is to deploy the application to the Kubernetes cluster. I will provide example manifests, you will just need to provide your dockerhub image, and if you are using a diffent ingress controller other than nginx, you will need to review those.

I am providing:

Deployment
Service
Ingress

First create the directory:

mkdir -p kubernetes

Then our kubernetes/deployment.yaml:

kubernetes/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: healthcheck-api
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: healthcheck-api
  template:
    metadata:
      labels:
        app: healthcheck-api
    spec:
      serviceAccountName: pod-reader
      containers:
      - name: healthcheck-api
        image: "<replace-me>" 
        ports:
        - name: http
          protocol: TCP
          containerPort: 8080
        env:
        - name: PORT
          value: "8080"

Then our kubernetes/service.yaml:

kubernetes/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: healthcheck-api
  namespace: default
spec:
  selector:
    app: healthcheck-api
  ports:
    - name: http
      protocol: TCP
      port: 8080
      targetPort: 8080
  type: ClusterIP

Then lastly our ingress kubernetes/ingress.yaml:

kubernetes/ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  labels:
    app.kubernetes.io/instance: healthcheck-api
    app.kubernetes.io/managed-by: kubectl
    app.kubernetes.io/name: healthcheck-api
    app.kubernetes.io/version: 0.0.1
  name: healthcheck-api
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: api.int.mydomain.tech
    http:
      paths:
      - backend:
          service:
            name: healthcheck-api
            port:
              name: http
        path: /
        pathType: ImplementationSpecific

Then we can go ahead and deploy these resources using:

kubectl apply -f kubernetes/

Testing our Application

First we will check our endpoint that returns the status code (/healthcheck/pods/status):

curl -i http://api.int.mydomain.tech/healthcheck/pods/status
HTTP/1.1 200 OK
Date: Sun, 21 Jul 2024 10:02:53 GMT
Content-Type: application/json
Content-Length: 15
Connection: keep-alive

{"status":200}

Then we can test the endpoint that returns the content (/healthcheck/pods):

curl -s http://api.int.sektorlab.tech/healthcheck/pods | jq .

And a snippet of the output should look something like this:

[
  {
    "metadata": {
      "name": "basic-api-c56754765-6p2bn",
      "generateName": "basic-api-c56754765-",
      "namespace": "default",
      "uid": "418e766a-0df3-4ebb-b0e8-b8d437367124",
      "resourceVersion": "160293618",
      "creationTimestamp": "2024-05-31T00:27:55Z",
      "labels": {
        "app": "basic-api",
        "pod-template-hash": "c56754765"
      },
      "ownerReferences": [
        {
          "apiVersion": "apps/v1",
          "kind": "ReplicaSet",
          "name": "basic-api-c56754765",
          "uid": "8afc1869-7c95-4540-90ed-825b510106db",
          "controller": true,
          "blockOwnerDeletion": true
        }
      ],

Thank You

Thanks for reading, if you like my content, feel free to check out my website, and subscribe to my newsletter or follow me at @ruanbekker on Twitter.

Linktree: https://go.ruan.dev/links
Patreon: https://go.ruan.dev/patreon