AHdark Blog

This blog will introduce you to how to automatically push Docker images to Google Artifact Registry (GAR) using GitHub CI's automated build repository.

The functionality of GitHub CI allows for automatic building of Docker images and pushing them to GAR when code is pushed to a GitHub repository. This automated process helps improve development efficiency while ensuring consistency and reliability of code and images during the release of new versions.

This article will provide you with detailed steps and guidelines to help you easily implement the process of automatically building and pushing Docker images.

Introduction to Related Services#

In this chapter, we will briefly introduce the concepts and characteristics of Google Artifact Registry and IAM's Service Account authorization model, so that readers can better understand how to use them to build and manage Docker images. We will also discuss the relationship between Artifact Registry and IAM Service Account, and how to create and configure Service Accounts to achieve secure and convenient authentication and authorization experiences when using Artifact Registry.

Google Artifact Registry#

Google Artifact Registry is a fully managed Docker image repository service provided by Google Cloud Platform (GCP), which helps developers better manage and share Docker images. Google Artifact Registry supports common package management tools such as Docker, Maven, npm, and NuGet, helping developers better integrate the application build and deployment process into the DevOps workflow, thereby improving development efficiency and software delivery speed.

Unlike other Docker image repository services, Google Artifact Registry is tightly integrated with GCP and can use GCP's authentication and authorization mechanisms to manage and protect access to images. This provides higher security and reliability guarantees for enterprise-level applications. Additionally, Artifact Registry supports image storage in multiple regions to provide better geographical flexibility and application availability.

With Google Artifact Registry, developers can easily upload, manage, and share Docker images. It also supports version management, allowing developers to manage images based on application versions and easily roll back to previous versions. Furthermore, Artifact Registry provides an easy-to-use UI interface, making it more intuitive for developers to manage and browse images.

In addition to basic image storage and management functions, Google Artifact Registry also offers some other advanced features, such as container image signing and high-speed concurrent pulls, to meet the needs of different application scenarios. Among these, the container image signing feature helps developers better protect the integrity and security of images, while high-speed concurrent pulls facilitate rapid deployment and scaling of large-scale applications.

In summary, Google Artifact Registry is a powerful fully managed Docker image repository service that is tightly integrated with GCP, supports image storage and management in multiple regions, and provides an easy-to-use UI interface and various advanced features to help developers better manage and share Docker images and accelerate the application build and deployment process.

GCP Service Account Authorization Model#

The Service Account authorization model in Google Cloud Platform (GCP) is an important form of GCP's authentication and authorization mechanism. It allows users to create a special account for applications so that the applications can access and manage GCP resources. Compared to other authorization methods, the Service Account model offers higher security and controllability, helping users better protect the confidential information and sensitive data of applications.

In GCP, each project has a default Service Account, and users can assign different roles to this Service Account to obtain the corresponding authorization when accessing GCP resources. For example, users can assign different roles such as "read," "write," and "edit" to the Service Account to restrict its access and management scope of resources. Additionally, users can create custom Service Accounts and assign them to specific applications or services for more granular authorization management.

Using the Service Account authorization model, users can access and manage GCP resources through various means, such as Google Cloud Console, gcloud command-line tool, and GCP API. Furthermore, Service Accounts also support standard authentication protocols such as OAuth2 and OpenID Connect, allowing users to better integrate Service Accounts into existing authentication and authorization mechanisms.

It is important to note that when using the Service Account authorization model, users need to configure appropriate roles and permissions for the Service Account and associate it with GCP resources.

In summary, the Service Account authorization model in Google Cloud Platform is a powerful and flexible authentication and authorization mechanism that can help users better manage and protect GCP resources. By assigning different roles and permissions to applications and associating them with GCP resources, users can achieve more granular authorization management, enhancing the security and reliability of applications.

Reproducing the Problem#

In Kubernetes, custom Docker images are a common way to package and deploy applications. When using custom Docker images, it is usually necessary to upload these images to a repository so that they can be deployed in a Kubernetes cluster. However, this approach also brings some security issues, such as the security, availability, and reliability of images.

1. The security of images is an important issue. If custom Docker images have vulnerabilities or insecure configurations, it may lead to attacks on applications or data breaches. Additionally, if the image repository does not have sufficient security measures, such as access control, authentication, and encryption, it may also be vulnerable to hacking or data leakage.
2. The availability and reliability of images are also significant concerns. If the image repository is unavailable or encounters errors during deployment, it may cause application deployment failures or prevent applications from running correctly. Therefore, it is necessary to use a reliable image repository and regularly back up and restore image data to ensure its availability and reliability.
3. Attention should be paid to access control and permission management of the image repository. If access control and permission management are not configured correctly, it may lead to unauthorized users or applications accessing or tampering with image data, resulting in security issues or application anomalies.

For these numerous issues, GAR provides good solutions. As a fully managed service of Google Cloud Platform (GCP), GAR offers a wealth of features and tools to help users better manage and protect Docker images, thereby improving the security, availability, and reliability of applications.

1. GAR provides comprehensive security features and mechanisms, such as access control, authentication, and encryption, to ensure the security and confidentiality of images. Users can use GCP's Identity and Access Management (IAM) and Service Account mechanisms to manage and authorize access permissions to images, and use tools like Container Analysis to identify and fix vulnerabilities and security risks in images.
2. GAR supports image storage in multiple regions and provides advanced features such as high-speed concurrent pulls and container image signing to meet the needs of different application scenarios. Users can choose suitable image storage regions based on the geographical location and performance requirements of applications, and use high-speed concurrent pulls to improve deployment and scaling efficiency. Container image signing helps users protect the integrity and security of images, preventing them from being tampered with or injected with malicious code.

GAR provides a wealth of features and tools to help users better manage and protect Docker images. By using GCP's authentication and authorization mechanisms, multiple regions for image storage, advanced features, and an easy-to-use interface, users can better protect and manage images, enhancing the security, availability, and reliability of applications.

Using CI for Automatic Push#

GitHub CI can be used to automate the building, testing, and pushing of Docker images, saving them to image repositories such as Google Artifact Registry (GAR). This automated process can greatly improve development efficiency and ensure consistency and reliability of code and images during the release of new versions.

CI Work Content#

Using GitHub CI to deploy to Google Artifact Registry (GAR) can help users better integrate the DevOps process and achieve automated building and deployment. Here are the main tasks for deploying GAR with GitHub CI:

1. Use actions/checkout to check out the repository
2. Use google-github-actions/auth to generate a temporary token for the Service Account
3. Use docker/login-action to log in to GAR
4. Generate a tag and push to Docker Registry

The above outlines the main content of GitHub CI, and what we need to do is configure OIDC and other authorization services for it in GCP.

Here, we choose to generate a temporary token through GitHub Action's OIDC rather than a persistent token mainly to avoid service information leakage due to data breaches, which reflects the commitment to service security.

Creating and Configuring Workload Identity Federation#

This section is adapted from the README of google-github-actions/auth, with some modifications.

Before that, you need to configure your gcloud CLI. You can log in using the gcloud auth login command: https://cloud.google.com/sdk/gcloud/reference/auth/login

For convenience in subsequent operations, we will define some reusable variables as temporary variables. These variables will be automatically deleted after the Bash session ends, so you don't need to worry. Note that this command can only be used on Linux systems, so you cannot follow the tutorial using cmd or other tools.

export PROJECT_ID=ci-project
export SERVICE_ACCOUNT=ci-service-account
export WORKLOAD_IDENTITY_POOL=ci-service-pool
export WORKLOAD_PROVIDER=github-provider

First, create or use an existing Google Cloud project. You must have permission to create workload identity pools, workload identity providers, and manage service accounts and IAM permissions.

We need to create a GCP Service Account. If you already have a Service Account, please note its email address and skip this step.

gcloud iam service-accounts create "${SERVICE_ACCOUNT}" \
  --project "${PROJECT_ID}" # Create Service Account

gcloud services enable iamcredentials.googleapis.com \
  --project "${PROJECT_ID}" # Allow using IAM API

Create a Workload Identity Pool to manage the roles of GitHub Action in the Google Cloud permission system:

gcloud iam workload-identity-pools create "${WORKLOAD_IDENTITY_POOL}" \
  --project="${PROJECT_ID}" \
  --location="global" \
  --display-name="${WORKLOAD_IDENTITY_POOL}"

Then, use the command to get the identifier of the Workload Identity Pool:

gcloud iam workload-identity-pools describe "${WORKLOAD_IDENTITY_POOL}" \
  --project="${PROJECT_ID}" \
  --location="global" \
  --format="value(name)"

# This should look like:
#
#   projects/123456789/locations/global/workloadIdentityPools/my-pool
#

Save the output as a new variable:

export WORKLOAD_IDENTITY_POOL_ID=whatever-you-got-back

Create an OIDC Provider that provides service for GitHub access within the pool:

gcloud iam workload-identity-pools providers create-oidc "${WORKLOAD_PROVIDER}" \
  --project="${PROJECT_ID}" \
  --location="global" \
  --workload-identity-pool="${WORKLOAD_IDENTITY_POOL}" \
  --display-name="${WORKLOAD_PROVIDER}" \
  --attribute-mapping="google.subject=assertion.sub,attribute.actor=assertion.actor,attribute.repository=assertion.repository" \
  --issuer-uri="https://token.actions.githubusercontent.com"

Allow GitHub Action based on your repository to log in to the service account through the provider:

export REPO=my-username/my-repo # Your repository name, case-sensitive

gcloud iam service-accounts add-iam-policy-binding "${SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com" \
  --project="${PROJECT_ID}" \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/${WORKLOAD_IDENTITY_POOL_ID}/attribute.repository/${REPO}"

Request Google to return the identifier of that provider:

gcloud iam workload-identity-pools providers describe "${WORKLOAD_PROVIDER}" \
  --project="${PROJECT_ID}" \
  --location="global" \
  --workload-identity-pool="${WORKLOAD_IDENTITY_POOL}" \
  --format="value(name)"

You will then receive an identifier that is different from the above WORKLOAD_IDENTITY_POOL_ID, which should be longer. You need to save this string, as we will use it in GitHub Action later.

Finally, we need to ensure that the service account created at the beginning has permission to operate Google Artifact Registry. Use the above command to add appropriate IAM roles and permissions to the service account so that it can access and manage resources in Google Artifact Registry:

gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member="serviceAccount:${SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com" \
    --role="roles/artifactregistry.admin"

To verify success, you can request Google to print out the permissions assigned to the service account:

gcloud projects get-iam-policy $PROJECT_ID \
    --flatten="bindings[].members" \
    --format='table(bindings.role)' \
    --filter="bindings.members:${SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com"

Creating a GAR Repository#

Go to the GAR console on Google Cloud Platform: https://console.cloud.google.com/artifacts

Select the button above to create a new repository, choose Docker, and specify the region; in this article, our example uses us-west-2.

Create a new Repository

Creating a GitHub Action Workflow#

You need to add a new YAML file in the .github/workflows folder; please remember to replace the information provided in the example.

name: Docker Release
on:
  push:

jobs:
  gar-push:
    name: Docker release to Google Artifact Registry
    runs-on: ubuntu-latest

    permissions:
      contents: "read"
      id-token: "write"

    steps:
      - id: checkout
        name: Checkout
        uses: actions/checkout@v3

      - id: auth
        name: Authenticate with Google Cloud
        uses: google-github-actions/auth@v1
        with:
          token_format: access_token
          workload_identity_provider: ${{ secrets.GCP_PROVIDER_ID }}
          service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}@${{ secrets.GCP_PROJECT_ID }}.iam.gserviceaccount.com
          access_token_lifetime: 300s

      - name: Login to Artifact Registry
        uses: docker/login-action@v2
        with:
          registry: ${{ secrets.GCP_REGISTRY_REGION }}-docker.pkg.dev
          username: oauth2accesstoken
          password: ${{ steps.auth.outputs.access_token }}

      - name: Get tag
        id: get-tag
        run: echo ::set-output name=short_ref::${GITHUB_REF#refs/*/}

      - id: docker-push-tagged
        name: Tag Docker image and push to Google Artifact Registry
        uses: docker/build-push-action@v4
        with:
          push: true
          tags:
            ${{ secrets.GCP_REGISTRY_REGION }}-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/${{ secrets.GCP_REGISTRY_ID }}/<your-docker-image-name>:${{ steps.get-tag.outputs.short_ref }}
            ${{ secrets.GCP_REGISTRY_REGION }}-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/${{ secrets.GCP_REGISTRY_ID }}/<your-docker-image-name>:latest

Once this file is created, it should trigger a GitHub Action. If this task successfully pushes, then the main task of this article is complete.

Using Images from GAR#

This section describes how to pull or push Docker images from Google Artifact Registry. Using images from GAR can facilitate the management and deployment of Docker images, improving the security, availability, and reliability of applications.

Pull / Push Using Docker#

Using images from GAR can be accomplished through the Docker client. First, users need to obtain GAR credentials (such as authentication tokens) from the GCP console to connect to GAR using the Docker client. Then, users can use the Docker client to pull or push Docker images to GAR.

Specifically, users need to use the following command to pull an image from GAR:

docker pull [REGION]-docker.pkg.dev/[PROJECT-ID]/[REPOSITORY-ID]/[IMAGE]:[TAG]

Where [REGION] is the GAR region, [PROJECT-ID] is the GCP project ID, [REPOSITORY-ID] is the GAR repository ID, [IMAGE] is the image name, and [TAG] is the image tag.

Users can also use the following command to push a Docker image to GAR:

docker push [REGION]-docker.pkg.dev/[PROJECT-ID]/[REPOSITORY-ID]/[IMAGE]:[TAG]

In this command, users need to specify the image to be pushed and its corresponding tag.

In summary, using images from GAR can be accomplished through the Docker client. By obtaining GAR credentials and using the Docker client to pull or push images, users can conveniently manage and deploy Docker images, enhancing the security, availability, and reliability of applications.

Using in GKE#

Since the Compute Engine roles used by the nodes in GKE have read-only access to GAR by default, you can directly pull images as unauthenticated Dockers.

You only need to fill in the image address in the container group's Image section, such as asia-east2-docker.pkg.dev/ahdark-services/ahdark-services/ahdark-blog, to pull the image normally.

Conclusion#

This article provides a detailed introduction on how to automatically push Docker images to GAR using GitHub Action and Google Artifact Registry.

First, we introduced the concepts and characteristics of GAR, as well as the Service Account authorization model in GCP.

Next, we detailed how to create and configure Workload Identity Federation and how to use GitHub CI to automatically push Docker images to GAR.

Finally, we provided steps on how to pull or push Docker images from GAR using the Docker client.

Through the guidelines and steps in this article, users can quickly and conveniently automatically push Docker images to GAR, achieving version management and sharing of images, while enhancing the security, availability, and reliability of applications. In actual application development, using the automated process of GitHub Action and GAR can greatly improve development efficiency and help users better implement the DevOps process.