CI/CD for GKE using Cloud Deploy
This tutorial will guide you through creating a CI/CD Pipeline on the Google Cloud Platform using Cloud Build and Cloud Deploy.
Cloud Deploy is the latest service from Google in the Continuous Delivery space and automates deployments of your application into your GKE clusters.
Under the covers it leverages Cloud Build for its job execution platform, integrates with Cloud Operations suite and Cloud Pub/Sub for further automation, and captures and reports on two of your key DORA Metrics.
Let’s take a look.
GCP Services used in this tutorial:
All the source code for this tutorial is available in GitHub.
Implementation Steps
In this guide we will go through the following steps.
- Build a containerized application using Cloud Build
- Set up GKE environments for Staging and Production
- Create the Kubernetes yaml mainfests for the Deployment and Service
- Write a Delivery Pipeline to deploy the application to GKE
- Trigger a new build and deployment to Staging after every successful build
Build a containerized application using Cloud Build
To begin we’re going to create a Hello World application and build it with Cloud Build. The generated container images will be pushed to Artifact Registry.
Create an application
For this demonstration we will use a Python Flask application, but you can use your go-to language if you prefer. If you’re following along with me, create main.py in your project folder. Here is my trivial Hello World application.
from flask import Flaskapp = Flask(__name__)
@app.route("/")def hello():
return "Hello World!"# run the app.
if __name__ == "__main__":
app.run()
Add requirements.txt to define the two dependencies.
Flask==2.0.2
gunicorn==20.1.0This should be sufficient to build and test locally. Now let’s automate…
Create a Docker repository in Artifact Registry
We need a repository in Artifact Registry to hold our container image. Artifact Registry can manage multiple repository formats, such as maven, npm, and in our case docker.
If this is your first time using Artifact Registry, then you will need to enable the artifactregistry.googleapis.com API.
Type the command into a Cloud Shell session as shown below.
gcloud artifacts repositories \
create my-docker-repo \
--repository-format=docker \
--location=us-central1 \
--description="Docker repository"Alternatively if you are using the console, head over to https://console.cloud.google.com/artifacts and hit “Create Repository”.
Configure Cloud Build
We’re going to be using Cloud Build to build our Hello World application, so create a cloudbuild.yaml file with the following contents.
steps:
- name: 'gcr.io/k8s-skaffold/pack'
entrypoint: 'pack'
args: ['build', '--builder=gcr.io/buildpacks/builder', '--publish', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-docker-repo/hello-world:v1']This example uses Buildpacks to build the python application (there is no need to write a Dockerfile) and saves the generated container image to Artifact Registry.
You will notice we have hardcoded as “v1” — later we will change this to the git commit SHA so that our end-to-end process is seamless and dynamic.
Final step before building, we also need to specify the command to run when the container is executed. Create a file called Procfile with a single line:
web: gunicorn --bind :8080 main:appTest your build using gcloud builds submit and you should find that the build runs successfully and your image is available in Artifact Registry.
Note that if this was your first time using Cloud Build, then you will need to enable the cloudbuild.googleapis.com API.
Create Kubernetes clusters in GKE
For this demonstration we are going to setup two GKE clusters, one for Staging and one for Production. We’re also going to use GKE Autopilot.
If you have not used GKE before, then you will need to enable the container.googleapis.com service first.
gcloud container clusters create-auto staging \
--region us-central1 \
--project=PROJECT_IDgcloud container clusters create-auto production \
--region us-central1 \
--project=PROJECT_ID
These commands may take a few minutes but after a wait you should have two running clusters ready for your application workloads to be deployed.
Create Kubernetes manifests
For our Hello World application we shall create two configurations, one for the Pod (wrapped into a Deployment) and one for the Service.
Create a subdirectory called k8s and create two files, k8s/deployment.yaml and k8s/service.yaml.
deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world-app
spec:
selector:
matchLabels:
app: hello-world-app
template:
metadata:
labels:
app: hello-world-app
spec:
containers:
- name: hello-world-app
image: hello-world-app
ports:
- containerPort: 8080service.yaml:
---
apiVersion: v1
kind: Service
metadata:
name: hello-world-service
spec:
ports:
- port: 80
targetPort: 8080
type: LoadBalancer
selector:
app: hello-world-app
You will notice how we reference the image as “hello-world-app”, that is because Skaffold will dynamically update the image later.
Use Cloud Deploy to deploy the application
Configure Delivery Pipeline in Cloud Deploy
Now that we have our application and its manifests in place, we can turn our attention to the Cloud Deploy configuration. Back in the root directory, create a file called clouddeploy.yaml.
We need to tell Cloud Deploy about the GKE clusters — a cluster is known to Cloud Deploy as a Target — and we provide the sequence of clusters/targets that our pipeline (DeliveryPipeline) should go through, in our case this will be “Staging” followed by “Production”.
---
apiVersion: deploy.cloud.google.com/v1beta1
kind: DeliveryPipeline
metadata:
name: hello-world-app
description: Hello World Deployment Pipeline
serialPipeline:
stages:
- targetId: staging
- targetId: production---
apiVersion: deploy.cloud.google.com/v1beta1
kind: Target
metadata:
name: staging
description: Staging Environment
gke:
cluster: projects/PROJECT_ID/locations/us-central1/clusters/staging---
apiVersion: deploy.cloud.google.com/v1beta1
kind: Target
metadata:
name: production
description: Production Environment
gke:
cluster: projects/PROJECT_ID/locations/us-central1/clusters/production
Apply the Cloud Deploy manifest before proceeding. If this is your first time you will need to enable the clouddeploy.googleapis.com service.
gcloud deploy apply --file=clouddeploy.yaml \
--region us-central1 \
--project=PROJECT_IDOn completion, you should have a Delivery Pipeline ready for releases.
Deploy application using the Delivery Pipeline
Cloud Deploy uses Skaffold for manifest rendering. We can use a simple skaffold.yaml configuration since we’re using Cloud Build for the build stage — there’s loads more to Skaffold though, including local developer integrations, so check out the references at the end to dig deeper.
apiVersion: skaffold/v2beta16
kind: Config
deploy:
kubectl:
manifests: ["k8s/*.yaml"]Let’s test this out manually at first — coming up we’ll update Cloud Build to automate everything end-to-end. Run the command below to create the initial Cloud Deploy Release to the Staging cluster.
gcloud deploy releases create initial-release-1 \
--delivery-pipeline=hello-world-app \
--region=us-central1 \
--source=./ \
--images=hello-world-app=us-central1-docker.pkg.dev/PROJECT_ID/my-docker-repo/hello-world:v1This uses Skaffold to render our manifests and kubectl to apply them to the cluster. With --images we tell Skaffold to replace any references to the “hello-world-app” image placeholder with the image we built earlier and pushed to Artifact Registry; this syntax allows us to supply mappings for all images and avoids the need to update the manifests prior to every deployment.
In the console, you will find details of your first release and you should see it successfully deployed to Staging.
If you head over to the GKE console, you will find the endpoint for your Staging environment. Click to verify your application is working as expected.
Promote the release to Production
Now that you are happy with the release and tested it in Staging, you are ready to promote it to Production. There are a number of ways to do this, but the most prominent is on the Pipeline visualisation page.
You can also re-deploy or roll back the release. Select “Promote release” to initiate the deployment to your Production cluster.
On successful completion of the release rollout, head back over to GKE to verify your application is now running in Production as expected.
Automating releases end-to-end
So far we have hardcoded the version in our build as “v1" and manually initiated the deployment; it would be good if we could make this more dynamic. We should also configure the pipeline to run after every push to git.
Firstly, let’s update the cloudbuild.yaml file to include the Git commit SHA to differentiate between artifacts. Cloud Build provides a substitution for builds invoked by triggers, so we’ll use that by switching v1 to $SHORT_SHA.
Next, we shall include the deployment as a follow-on step by using the cloud-sdk tool — the command is the same as we ran manually earlier. The updated configuration can be viewed below (changes are highlighted).
steps:
- name: 'gcr.io/k8s-skaffold/pack'
entrypoint: 'pack'
args: ['build', '--builder=gcr.io/buildpacks/builder', '--publish', 'us-central1-docker.pkg.dev/$PROJECT_ID/my-docker-repo/hello-world:$SHORT_SHA']
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
entrypoint: 'bash'
args:
- '-c'
- >
gcloud deploy releases create release-$BUILD_ID
--delivery-pipeline=hello-world-app
--region=us-central1
--source=./
--images=hello-world-app=us-central1-docker.pkg.dev/$PROJECT_ID/my-docker-repo/hello-world:$SHORT_SHANow let’s commit all these files, push them to GitHub, and hook GitHub to Cloud Build so that future changes are automatically built and released.
Set up automated build Trigger
Once you have your source code in GitHub, we want to set up Cloud Build to trigger a new build every time there is new code pushed to the repository.
Head over to the Cloud Build console, and go into the “Triggers” section and from there click on “Create Trigger”.
Give the trigger a name such as “hello-world”, and under “Source” you should connect your GitHub source code repository.
The rest of the settings should remain as their defaults, specifically that Cloud Build will use the cloudbuild.yaml file as its build definition file.
Run a new build and deployment
Once your “Trigger” has been created you can manually test it out by clicking the “Run” button in the console.
Your build may fail with a permission error; to resolve it, grant the “Cloud Deploy Releaser” role to your Cloud Build Service Account.
On successful completion of the build, you should find a new Release in Cloud Deploy and a successful deployment to Staging. Note the dynamic release name using the Cloud Build number, and your container image manifests tags include the SHA from the git commit.
Make a code change and see it roll all the way through
Final step. We now want to verify that a source code change triggers Cloud Build to generate a new container image and for Cloud Deploy to push that new release to our Staging environment. To mock this for the demonstration, I’ve made a minor change to the “Hello World” message and pushed a new commit to GitHub. Cloud Build detects the change immediately and begins to roll out the new version of the application. After a few minutes, you should see the next version of your application running in Staging.
Well done if you followed along this far — that’s a wrap.
DORA Metrics
Earlier I mentioned Cloud Deploy provides a couple of DORA Metrics out of the box. These are Deployment frequency and Deployment failure rate.
Deployment frequency is how often does the delivery pipeline deploy to the final target in a delivery pipeline. Deployment failure rate provides the percentage of failed rollouts to the final target in a delivery pipeline. The “final target” would typically be production.
These are two of the four key metrics defined by the DevOps Research and Assessment (DORA) program.
Conclusion
In this tutorial we automated the release of a containerized application to GKE using Cloud Build and Cloud Deploy, along with Artifact Registry for storing our images, and GitHub for source code. We configured our GKE clusters into a sequential Continuous Delivery pipeline, Staging > Production, and automatically generated and deployed new releases after every commit.
All the source code for this tutorial is available in GitHub.
