CircleCI, Terraform, and GKE Deployment
How LZStock automates cloud infrastructure via Terraform and securely deploys microservices to Google Kubernetes Engine (GKE) using CircleCI and Helm.
- 100% Immutable Infrastructure (IaC): Eradicated "Configuration Drift" by utilizing Terraform to programmatically provision the entire GCP footprint (VPC, Cloud SQL, GKE), ensuring all infrastructure changes are version-controlled, auditable, and instantly reproducible.
- Zero-Trust CI/CD Pipelines: Mitigated supply-chain attack vectors by applying the Principle of Least Privilege (PoLP). CircleCI relies on strictly-scoped Kubernetes RBAC ServiceAccounts, possessing only the granular permissions required to deploy application Helm charts, completely isolated from underlying node infrastructure.
- Stateful Workloads & Auth Sidecars: Navigated complex Kubernetes state management by utilizing PersistentVolumeClaims (PVCs) for internal Redis deployments, and seamlessly integrated the Cloud SQL Auth Proxy sidecar to guarantee IAM-authenticated, TLS-encrypted database connections without exposing public IPs.
The Objective
Clicking through the Google Cloud Console to create a database or Kubernetes cluster is an anti-pattern that leads to "Configuration Drift" and fragile infrastructure. Furthermore, giving a CI/CD pipeline (like CircleCI) full cluster-admin access exposes the entire company to a supply-chain attack.
The objective is to establish a 100% Infrastructure-as-Code (IaC) foundation using Terraform, coupled with a highly restricted, automated CI/CD pipeline. This ensures that every network route, database instance, and microservice deployment is auditable, repeatable, and securely isolated.
The Mental Model & Deployment Toplogy
Our deployment workflow is strictly unidirectional. Developers push code, CircleCI builds and pushes artifacts to the GitLab Registry, and finally, CircleCI applies the Helm charts to the GKE Cluster using a strictly scoped RBAC token.
The Architectural Boundary:
- The Entrypoint: Engineers trigger changes through strictly separated operational boundaries. Infrastructure changes require explicit Terraform execution, while daily business logic updates are seamlessly handled via standard Git operations.
- Terraform (The Foundation): Responsible for slow-moving, stateful resources. It provisions the VPC, Cloud SQL, GKE cluster, and directly bootstraps the foundational middleware (NATS, Redis) into the cluster.
- CircleCI (The Application): Responsible for fast-moving, stateless microservices. It only possesses the permissions necessary to update the specific LZStock application Pods, completely isolated from the underlying infrastructure.
Core Implementation
The infrastructure is broken down into three lifecycle phases: Provisioning, Security Bootstrapping, and Application Delivery.
Infrastructure Provisioning (Terraform)
We manage the entire GCP footprint via Terraform. This includes the VPC, Subnets, Cloud NAT (to allow private GKE nodes to pull external images), the GKE Cluster, and the Cloud SQL PostgreSQL instance.
Crucially, Terraform also provisions the IAM Service Accounts and Secrets required for the GKE Pods to securely authenticate with Cloud SQL without hardcoding database passwords.
CI/CD Security Boundary (CircleCI to GKE)
To allow CircleCI to deploy our Helm charts, it needs a token. Instead of sharing a global admin key, we apply the Principle of Least Privilege (PoLP) by creating a dedicated ServiceAccount bounded to a specific namespace (lzstock), restricted via a RoleBinding.
# 1. Create a dedicated CI/CD identity
kubectl create serviceaccount circleci-deploy -n lzstock
# 2. Bind it ONLY to deployment-related permissions (No access to cluster nodes/secrets outside its scope)
kubectl create clusterrolebinding circleci-deploy-binding \
--clusterrole=circleci-deploy-cluster-admin \
--serviceaccount=lzstock:circleci-deploy
# 3. Generate a long-lived token for CircleCI environment variables
kubectl create token circleci-deploy -n lzstock --duration=8760h
Stateful Workloads & Application Delivery (Helm)
While the API gateways are stateless, we deploy Redis internally via Kubernetes manifests. For the LZStock applications, we utilize Helm to template the environment variables and inject database credentials dynamically from Kubernetes Secrets.
# lzstock-deployment.yaml (Helm Template Snippet)
containers:
- name: lzstock-api
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
# Securely injecting Cloud SQL credentials provisioned by Terraform
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-credentials
key: password
Edge Cases & Trade-offs
- Connecting to Cloud SQL (Public IP vs. Private IP vs. Auth Proxy): We do not expose our PostgreSQL database to the public internet. While we could use VPC Peering (Private IP) between GKE and Cloud SQL, LZStock utilizes the Cloud SQL Auth Proxy running as a sidecar container. The trade-off is slightly more memory usage per Pod, but it provides automatic IAM authentication, end-to-end TLS encryption, and eliminates the need to manage static IP whitelists.
- PersistentVolumeClaim (PVC) vs. PersistentVolume (PV): When deploying Redis on K8s, developers often confuse these two. The trade-off is abstraction. A PV is the actual physical disk in GCP (e.g., a 10GB SSD). A PVC is the Pod's request for storage. We only write the PVC in our manifests. Kubernetes' dynamic provisioner intercepts the PVC and automatically provisions the underlying PV in GCP, decoupling our code from cloud-specific hardware IDs.
- StatefulSet vs. Deployment (The Redis Dilemma): We deployed Redis using a Deployment with a PVC. Is this an anti-pattern?
- If this were a multi-node Redis Cluster requiring stable network identities (redis-0, redis-1) and ordered replication, a StatefulSet would be strictly mandatory.
- However, for a single-node Redis acting purely as an ephemeral cache, a standard Deployment is an acceptable trade-off that simplifies operations, as we don't need strict pod-identity guarantees.
- The Long-Lived Token Risk: In step 3.2, we generated an 8760-hour (1 year) static token for CircleCI. The trade-off here is convenience versus security. Static tokens can leak. The next architectural evolution for LZStock is moving to GCP Workload Identity Federation (OIDC), completely eliminating static tokens by allowing CircleCI to dynamically exchange its GitHub/GitLab identity for short-lived GCP access tokens.
Appendix: Infrastructure Blueprints
Expand to view the Bootstrap Scripts and K8s Configs
- GCP Bootstrap (Pre-Terraform)
# Authenticate and set project
gcloud auth login
gcloud config set project $TF_VAR_project_id
# Enable necessary APIs for Terraform
gcloud services enable serviceusage.googleapis.com
gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable iam.googleapis.com
gcloud services enable container.googleapis.com
gcloud services enable compute.googleapis.com
gcloud services enable sqladmin.googleapis.com
# Link Billing
gcloud billing projects link $TF_VAR_project_id --billing-account=$TF_VAR_billing_account_id
- Kubeconfig Generation for CircleCI
apiVersion: v1
kind: Config
current-context: ${CLUSTER_NAME}
contexts:
- name: ${CLUSTER_NAME}
context:
cluster: ${CLUSTER_NAME}
user: circleci-deploy
clusters:
- name: ${CLUSTER_NAME}
cluster:
certificate-authority-data: ${CLUSTER_CA_CERT}
server: ${CLUSTER_ENDPOINT}
users:
- name: circleci-deploy
user:
token: ${SA_SECRET_TOKEN}
- Redis Kubernetes Manifests
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: redis-pvc
namespace: redis
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
storageClassName: standard
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: redis
spec:
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:7.2-alpine
volumeMounts:
- name: redis-data
mountPath: /data
volumes:
- name: redis-data
persistentVolumeClaim:
claimName: redis-pvc