Automate GKE Deployments with ArgoCD & GitHub

The Mission: Automate the entire software supply chain. We are moving from manual kubectl updates to a fully automated "Commit-to-Cluster" pipeline.

Welcome back to the Building a Production-Grade SRE Platform on Kubernetes series.

If you are just joining us, we are building a production-grade Kubernetes platform from scratch. Here is the journey so far:

[Part 1: The Foundation] – We provisioned a cost-optimized GKE cluster using Terraform.
[Part 2: The Engine] – We established a secure GitOps workflow with ArgoCD to manage our cluster state and used Gateway API for external access.
[Part 3: The Eyes] – We deployed the LGTM Observability stack (Loki, Grafana, Tempo, Prometheus) to see exactly what our microservices are doing.

But something is missing.

Right now, we have a Ferrari engine (ArgoCD) and a high-tech dashboard (Grafana), but we are still pushing the car by hand. Every deployment requires us to build a Docker image locally, tag it, and manually edit a YAML file in our Git repo.

That stops today.

In Part 4, we build "The Factory." We are going to automate the entire software supply chain - from a git push on your laptop to a rolling update in the cloud, without ever touching kubectl or managing a single dangerous Service Account Key.

The Strategy:

Security First: No Artifact Registry credentials. We use Workload Identity Federation (OIDC) for GitHub Actions to push images securely.
Immutable Artifacts: Using Distroless Docker images for a minimal security footprint.
Automated Write-Back: Implementing ArgoCD Image Updater to watch the registry and commit version changes back to Git, keeping the "Git as Source of Truth" promise intact.

1. The Foundation: Infrastructure as Code

Before writing code, we need a place to store it and a secure way to push it. We use Terraform to build the Artifact Registry and the "Keys" (Workload Identity).

The Terraform Setup

We created two key resources to avoid generating dangerous JSON Service Account keys:

The Private Registry (registry.tf): Creates a secure Google Artifact Registry (GAR) repository.

resource "google_project_service" "artifact_registry" {
  service            = "artifactregistry.googleapis.com"
  disable_on_destroy = false
}

resource "google_artifact_registry_repository" "my_repo" {
  location      = var.region
  repository_id = "my-artifact-repo"
  description   = "Docker repository for apps"
  format        = "DOCKER"

  depends_on = [google_project_service.artifact_registry]
}

Keyless Auth (github-oidc.tf): Configures a Workload Identity Pool that trusts your specific GitHub repository. This allows the GitHub Actions runner to impersonate a GCP Service Account temporarily.

# 1. Service Account for GitHub Actions
resource "google_service_account" "github_actions" {
  account_id   = "github-actions-sa"
  display_name = "Service Account for GitHub Actions"
}

# 2. Allow SA to PUSH to Artifact Registry
resource "google_artifact_registry_repository_iam_member" "github_push" {
  project    = google_artifact_registry_repository.my_repo.project
  location   = google_artifact_registry_repository.my_repo.location
  repository = google_artifact_registry_repository.my_repo.name
  role       = "roles/artifactregistry.writer"
  member     = "serviceAccount:${google_service_account.github_actions.email}"
}

# 3. Workload Identity Pool for GitHub
resource "google_iam_workload_identity_pool" "github_pool" {
  workload_identity_pool_id = "github-pool"
  display_name              = "GitHub Actions Pool"
}

resource "google_iam_workload_identity_pool_provider" "github_provider" {
  workload_identity_pool_id          = google_iam_workload_identity_pool.github_pool.workload_identity_pool_id
  workload_identity_pool_provider_id = "github-provider"
  display_name                       = "GitHub Actions Provider"

  attribute_mapping = {
    "google.subject"       = "assertion.sub"
    "attribute.actor"      = "assertion.actor"
    "attribute.repository" = "assertion.repository"
  }

  attribute_condition = "assertion.repository == \"${var.github_repo_name}\""

  oidc {
    issuer_uri = "https://token.actions.githubusercontent.com"
  }
}

# 4. Bind the Pool to the Service Account
resource "google_service_account_iam_member" "github_oidc_binding" {
  service_account_id = google_service_account.github_actions.name
  role               = "roles/iam.workloadIdentityUser"
  member             = "principalSet://iam.googleapis.com/${google_iam_workload_identity_pool.github_pool.name}/attribute.repository/${var.github_repo_name}"
}

Note: I’m not going to walk through every Terraform resource line-by-line here. The important takeaway is that no static keys are ever generated - GitHub Actions authenticates using OIDC and temporary credentials.

Apply the Infrastructure:

# Grant terraform SA, permission to to manage WorkloadIdentity
gcloud projects add-iam-policy-binding sre-portfolio-platform \
    --member="serviceAccount:terraform-deployer@sre-portfolio-platform.iam.gserviceaccount.com" \
    --role="roles/iam.workloadIdentityPoolAdmin"

# Grant terraform SA, permission to create/manage Artifact Registry Repositories
gcloud projects add-iam-policy-binding sre-portfolio-platform \
    --member="serviceAccount:terraform-deployer@sre-portfolio-platform.iam.gserviceaccount.com" \
    --role="roles/artifactregistry.admin"

# Initialize and apply
terraform init
terraform apply

2. The Product: A Minimal Cloud-Native Go App

We built a lightweight Go web server to act as our payload. Why Go? Because it compiles into a single static binary, making it perfect for the secure Distroless container strategy we are using.

The Structure:

src/go-app/
├── main.go       # The web server (returns JSON version info)
└── Dockerfile    # Multi-stage build (Distroless final image)

src/go-app/main.go:

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "os"
    "time"
)

type Response struct {
    Message   string    `json:"message"`
    Timestamp time.Time `json:"timestamp"`
    Hostname  string    `json:"hostname"`
    Version   string    `json:"version"`
}

func handler(w http.ResponseWriter, r *http.Request) {
    hostname, _ := os.Hostname()
    resp := Response{
        Message:   "Hello from the Go Application!",
        Timestamp: time.Now(),
        Hostname:  hostname,
        Version:   "v1.0.0",
    }

    log.Printf("Received request from %s", r.RemoteAddr)
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(resp)
}

func main() {
    http.HandleFunc("/", handler)
    port := "8080"
    fmt.Printf("Starting server on port %s...\n", port)
    if err := http.ListenAndServe(":"+port, nil); err != nil {
        log.Fatal(err)
    }
}

It is a basic web (HTTP) server serving at port 8080 with an initial version of 1.0.0. To test the Zero-Touch flow, we simply change that string to Version: "v1.1.0" and push. The automation handles the rest.

The Dockerfile Strategy: We used a multi-stage build. The Builder stage compiles the Go binary, and the Final stage copies only that binary into a gcr.io/distroless/static image. This results in a tiny, secure image with no OS tools for attackers to use.

# Stage 1: Build
FROM golang:1.25-alpine AS builder
WORKDIR /app
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o main .

# Stage 2: Distroless Runtime
FROM gcr.io/distroless/static-debian12
WORKDIR /
COPY --from=builder /app/main .
EXPOSE 8080
ENTRYPOINT ["/main"]

3. The Pipeline: CI with GitHub Actions

With the infrastructure ready, we defined the CI workflow.

.github/workflows/build-push.yaml:

name: Build and Push
on:
  push:
    paths: ['src/go-app/**'] # Only run when app code changes

env:
  GAR_LOCATION: us-central1-docker.pkg.dev/sre-portfolio-platform/my-artifact-repo
  WI_PROVIDER: projects/99833001104/.../providers/github-provider # From Terraform
  WI_SERVICE_ACCOUNT: github-actions-sa@sre-portfolio-platform.iam.gserviceaccount.com

jobs:
  build-push:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write # CRITICAL: Required for OIDC authentication

    steps:
      - uses: actions/checkout@v4

      # 1. Authenticate to Google Cloud (No JSON Keys!)
      - id: auth
        uses: google-github-actions/auth@v2
        with:
          token_format: 'access_token'
          workload_identity_provider: ${{ env.WI_PROVIDER }}
          service_account: ${{ env.WI_SERVICE_ACCOUNT }}

      # 2. Login to Artifact Registry using the temporary OIDC token
      - uses: docker/login-action@v3
        with:
          registry: us-central1-docker.pkg.dev
          username: oauth2accesstoken
          password: ${{ steps.auth.outputs.access_token }}

      # 3. Build and Push with Git SHA tag
      - uses: docker/build-push-action@v5
        with:
          context: ./src/go-app
          push: true
          tags: |
            ${{ env.GAR_LOCATION }}/go-app:${{ github.sha }}
            ${{ env.GAR_LOCATION }}/go-app:latest

This workflow does the heavy lifting:

Auth: Authenticates via OIDC (using the wi_provider_name from Terraform).
Build: Builds the Docker image.
Push: Pushes the artifact to Google Artifact Registry.

Local Testing with act: Before pushing to GitHub, we tested the pipeline locally to save time:

brew install act
act -W .github/workflows/build-push.yaml --container-architecture linux/amd64

Note: The build-push stage fails, because the local act binary cannot emulate the actual Workload Identity credential, but rest of the stages worked fine, so we shall push the code now.

4. The Deployment: ArgoCD

We defined the Kubernetes manifests to run the app.

The Manifests (kubernetes/deployments/go-app/):

deployment.yaml: The pod definition.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-app
  namespace: go-app
  annotations:
    # This annotation tells ArgoCD Image Updater to write back to THIS file
    argocd-image-updater.argoproj.io/write-back-method: git
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-app
  template:
    metadata:
      labels:
        app: go-app
    spec:
      containers:
      - name: go-app
        image: us-central1-docker.pkg.dev/sre-portfolio-platform/my-artifact-repo/go-app:latest
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: "64Mi"
            cpu: "100m"
          limits:
            memory: "128Mi"
            cpu: "200m"

service.yaml: Internal networking.

apiVersion: v1
kind: Service
metadata:
  name: go-app
  namespace: go-app
spec:
  selector:
    app: go-app
  ports:
  - port: 80
    targetPort: 8080
  type: ClusterIP

route.yaml: Exposed via Gateway API (public URL: http://api.techtalkswithanant.online).

kind: HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: go-app-route
  namespace: go-app
spec:
  parentRefs:
  - name: external-gateway
    namespace: default
  hostnames:
  - "api.techtalkswithanant.online"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: go-app
      port: 80

healthcheck.yaml: To allow Gateway to check the health of it’s backends.

apiVersion: networking.gke.io/v1
kind: HealthCheckPolicy
metadata:
  name: go-app-health-check
  namespace: go-app
spec:
  default:
    checkIntervalSec: 10
    timeoutSec: 5
    healthyThreshold: 1
    unhealthyThreshold: 2
    config:
      type: HTTP
      httpHealthCheck:
        port: 8080        
        requestPath: /    
  targetRef:
    group: ""
    kind: Service
    name: go-app

The GitOps Hook: Why Kustomize?

You might wonder: Why did we add a kustomization.yaml file?

In a rigid GitOps setup, your deployment.yaml is static. But CI/CD is dynamic - we generate new image tags (v1.0.1, v1.0.2) every few minutes. We need a clean way to update just the image tag without rewriting the entire deployment manifest.

The Solution: Kustomize acts as a mutable "pointer."

kubernetes/deployments/go-app/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - deployment.yaml  # The static structure
  - service.yaml
  - route.yaml
  - healthcheck.yaml

# The "Hook" for the Image Updater
images:
  - name: us-central1-docker.pkg.dev/sre-portfolio-platform/my-artifact-repo/go-app
    newTag: latest  # <--- THIS is what the bot edits

How it works:

The ArgoCD Image Updater scans your registry and finds a new tag (e.g., v1.1.0).
It clones your repo and looks specifically for a Kustomize definition matching the image name.
It commits a change only to this kustomization.yaml file, updating newTag: latest to newTag: v1.1.0.
ArgoCD detects the change, renders the final manifest (injecting the new tag into the Deployment), and syncs the cluster.

This separates the Structure (Deployment) from the State (Version), keeping your manifests clean and your diffs readable.

We registered this app with ArgoCD using an Application manifest (kubernetes/apps/go-app.yaml). Once synced, the app went live immediately.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: go-app
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
  annotations:
    argocd-image-updater.argoproj.io/image-list: go-app=us-central1-docker.pkg.dev/sre-portfolio-platform/my-artifact-repo/go-app
    argocd-image-updater.argoproj.io/go-app.update-strategy: newest-build
    argocd-image-updater.argoproj.io/write-back-method: git
spec:
  project: default
  source:
    repoURL: https://github.com/anantvaid/otel-platform-infra.git
    targetRevision: main
    path: kubernetes/deployments/go-app # Pointing to the manifests we just made
  destination:
    server: https://kubernetes.default.svc
    namespace: go-app
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Verification:
We shall see a go-app application deployed now in ArgoCD dashboard.

5. Installing the Engine: ArgoCD Image Updater

Before we can automate anything, we need to install the argocd-image-updater controller. Consistent with our App-of-Apps pattern, we don't run helm install manually. Instead, we define it as code.

The Wrapper Chart

We created a local wrapper chart to manage our configuration cleanly.

kubernetes/platform/image-updater/Chart.yaml

apiVersion: v2
name: argocd-image-updater
version: 1.0.4
dependencies:
  - name: argocd-image-updater
    version: 0.11.0
    repository: https://argoproj.github.io/argo-helm

The Workload Identity Script

Standard registry authentication usually requires a static password. To use GKE Workload Identity (and avoid keys), we need to teach the Updater how to fetch a Google Cloud token dynamically.

We achieved this by injecting a custom script into the Helm values.

kubernetes/platform/image-updater/values.yaml

argocd-image-updater:
  serviceAccount:
    create: true
    name: "argocd-image-updater"
    annotations:
      # Binds Kubernetes SA to Google SA
      iam.gke.io/gcp-service-account: "image-updater-sa@sre-portfolio-platform.iam.gserviceaccount.com"

  config:
    registries: 
      - name: us-central1-docker.pkg.dev
        api_url: https://us-central1-docker.pkg.dev
        prefix: us-central1-docker.pkg.dev
        ping: yes
        # Tell the updater to use our custom script
        credentials: ext:/scripts/gcp-auth.sh
        credsexpire: 55m

  authScripts:
    enabled: true
    scripts:
      # The script that fetches the OIDC token from GKE Metadata Server
      gcp-auth.sh: |
        #!/bin/sh
        TOKEN=$(wget -q -O - --header "Metadata-Flavor: Google" \
          http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token \
          | grep -o '"access_token":"[^"]*"' \
          | cut -d'"' -f4)
        echo "oauth2accesstoken:$TOKEN"

Why this matters: This script allows the pod to phone home to the GKE Metadata Server, exchange its identity for a valid Google Access Token, and login to the Artifact Registry, all without a single stored secret.

This is the most advanced part of the setup. If you’ve never worked with GKE Workload Identity before, this section may feel dense - that’s expected.

GitOps Registration

Finally, we added it to our bootstrap root-app.yaml so ArgoCD manages it.

File: kubernetes/bootstrap/updater.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: image-updater
  namespace: argocd
spec:
  project: default
  source:
    path: kubernetes/platform/image-updater
    repoURL: https://github.com/anantvaid/otel-platform-infra.git
    targetRevision: main
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd

6. The Automation

This was the complex ‘Senior Engineer’ challenge. We needed a bot to watch the registry and update Git automatically, but it had to be secure.

Challenge 1: The Bot's Identity

The Image Updater needs to read from Google Artifact Registry. We used Workload Identity (binding a Kubernetes Service Account to a Google Service Account) so the pod can authenticate without static keys.

Challenge 2: Git Write-Back Access

The bot needs to git push changes to your private repo. ArgoCD needs GitHub token credentials for this.

Solution:

Generated a GitHub PAT (Repo scope).
Stored it in GCP Secret Manager.
Synced it to Kubernetes using External Secrets Operator.

Critical Step: We templated the External Secret to include the specific label argocd.argoproj.io/secret-type: repository. Without this label, ArgoCD ignores the credentials.

kubernetes/platform/external-secrets/github-pat.yaml:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: github-pat
  namespace: argocd
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: gcp-secret-store
    kind: ClusterSecretStore
  target:
    name: git-creds
    creationPolicy: Owner
    template:
      metadata:
        labels:
          argocd.argoproj.io/secret-type: repository
      data:
        url: "https://github.com/anantvaid/otel-platform-infra.git"
        username: "anantvaid"
        password: "{{ .password }}"
  data:
  - secretKey: password
    remoteRef:
      key: github-pat
      version: latest

The "Handshake" (Annotations): We told the Updater which strategy to use in the Application manifest:

metadata:
  annotations:
    argocd-image-updater.argoproj.io/image-list: go-app=us-central1-docker.pkg.dev/.../go-app
    argocd-image-updater.argoproj.io/go-app.update-strategy: newest-build
    argocd-image-updater.argoproj.io/write-back-method: git

7. The Zero-Touch Test

The moment of truth: A full "Commit-to-Cluster" loop without touching kubectl.

The Change: I bumped the version in main.go from v1.0.0 to v1.1.0.
The Trigger: git push origin main.

The Chain Reaction:

GitHub Actions: Built and pushed the new image.
Image Updater: Detected the new tag in the registry.
Git Write-Back: The updater committed the change to my kustomization.yaml automatically.
ArgoCD: Synced the change to the cluster.

Result: Within 2 minutes, the live API updated automatically.

# Check the argocd-image-updater pod logs to see if it detects the changes in Registry
kubectl logs -n argocd -l app.kubernetes.io/name=argocd-image-updater -f

We see that argocd-image-updater automatically detects the new tag and commits the change directly to our kustomization.yamlfile in GitHub.

while true; do curl -s https://api.techtalkswithanant.online | grep version; sleep 2; done
# Output changes from "v1.0.0" -> "v1.1.2" automatically!

Summary: We have successfully built a factory that takes raw code and turns it into a running service securely and automatically. We eliminated manual deployments and static cloud keys, significantly improving our security posture and developer velocity.

You can find the code repository here: https://github.com/anantvaid/otel-platform-infra

Coming up next: Phase 5

The Guardrails – Policy as Code with Kyverno

We have built a high-speed "Factory" (Phase 4), but speed without safety is a disaster waiting to happen. What if a developer deploys a container running as root? What if someone accidentally exposes a private service to the public internet?

In Phase 5, we will implement Kyverno to enforce Kubernetes Policing. We will move from "trusting" our manifests to validating and mutating them automatically.

Block insecure deployments (e.g., no privileged containers).
Enforce best practices (e.g., require resource limits).
Mutate resources on the fly (e.g., inject sidecars automatically).

We are turning our cluster from a "Wild West" into a governed platform. See you in the next one! 👋

The CI/CD Factory: Zero-Touch GKE Deployments with ArgoCD & GitHub Actions

1. The Foundation: Infrastructure as Code

The Terraform Setup

2. The Product: A Minimal Cloud-Native Go App

3. The Pipeline: CI with GitHub Actions

4. The Deployment: ArgoCD

5. Installing the Engine: ArgoCD Image Updater

The Wrapper Chart

The Workload Identity Script

GitOps Registration

6. The Automation

Challenge 1: The Bot's Identity

Challenge 2: Git Write-Back Access

7. The Zero-Touch Test

Coming up next: Phase 5

Comments

Building a Production-Grade SRE Platform on Kubernetes

Zero Trust Security in Kubernetes: Kyverno & Istio Ambient

More from this blog

LangChain, LangGraph, LangSmith, and LangFlow - Actually Explained

From FastAPI to Cloud Run - Deploying AI Agents the Google Way

Stop Doing Manual SRE Reviews: Build an AI Auditor with GitHub MCP

Chaos Engineering - Proving Resilience in Kubernetes Platform

FinOps in Kubernetes - Taming the Cloud Bill with Kubecost

Command Palette

1. The Foundation: Infrastructure as Code

The Terraform Setup

2. The Product: A Minimal Cloud-Native Go App

3. The Pipeline: CI with GitHub Actions

4. The Deployment: ArgoCD

5. Installing the Engine: ArgoCD Image Updater

The Wrapper Chart

The Workload Identity Script

GitOps Registration

6. The Automation

Challenge 1: The Bot's Identity

Challenge 2: Git Write-Back Access

7. The Zero-Touch Test

Coming up next: Phase 5

Comments

Building a Production-Grade SRE Platform on Kubernetes

Zero Trust Security in Kubernetes: Kyverno & Istio Ambient

More from this blog