In this blog post, I share a setup of Gradle build cache nodes, which you may use to reduce your project build time. If you are part of a smaller team using non-enterprise Gradle projects, the contents of this article will be especially helpful.

As a build engineer, I encountered several challenges with Gradle build caching that demanded a simple, cost-effective alternative to Gradle Enterprise and provided me with:

  •  A reduction in build times through efficient caching.
  •  Ease of scaling with project growth.
  •  Build consistency across environments.
  •  A budget-friendly solution.
  • Full control over my caching infrastructure.

Please note that the outlined implementation deployed to AKS was suited to our organization (Eficode) but may be easily adjusted to another platform.

What is a Gradle build cache?

A Gradle build cache is a powerful feature that stores and reuses the output of previous builds. It significantly reduces the time for subsequent builds, which is especially beneficial in CI/CD environments where multiple occur.

Project overview

Our project provides a robust and scalable solution for deploying Gradle build cache nodes in AKS. You can find it open-sourced here. Key components include:

  • A Kubernetes StatefulSet with two replicas of the build cache node.
  • A Kubernetes Service to expose the cache.
  • An Ingress Controller for external access.
  • An AzureFile for persistent storage.

Prerequisites

To implement this solution, you'll need:

  • An Azure Kubernetes Service cluster.
  • Kubernetes CLI (kubectl).
  • Azure CLI (az).
  • Docker CLI.

Implementation steps

We chose AzureFile for persistent volume as we found this the simplest solution for an AKS. You may wish to consider an alternative depending on the needs of your project, team, or organization.

Setting up Azure file share

We started by creating an Azure file share to provide persistent storage for our cache nodes. This involved generating a storage account, a file share, and setting up the necessary Kubernetes secrets.

# Create a storage account
az storage account create --resource-group  --name  --location  --sku Standard_LRS

# Create a file share
az storage share create --account-name  --name 

# Get storage account key
az storage account keys list --resource-group  --account-name 

# Create Kubernetes secret
kubectl create secret generic azure-build-cache-secret --from-literal=azurestorageaccountname= --from-literal=azurestorageaccountkey= -n build-cache
Deploying the StatefulSet and service

Next, we deployed the StatefulSet and service using Kubernetes manifests. The StatefulSet ensured that our cache nodes had stable network identities and persistent storage.

kubectl apply -f gradle-cache-stateful-set.yaml
kubectl apply -f service.yaml

Configuring Ingress

We set up an Ingress resource to manage external access to our cache nodes. This allowed our developers to interact with the cache from outside the Kubernetes cluster. In the example, we used: webapprouting.kubernetes.azure.com as the Ingress class for simplicity, as we did not have strict requirements on how to access the cache nodes, whereas you might prefer a proper HTTPS server.

kubectl apply -f ingress.yaml

To restrict/configure user access, you can provide a custom configuration to «config-dir»/config.yaml at the node, but this comes with some challenges.

We wanted a preconfigured user access from the beginning with the config.yaml file set as a secret. If you want to mount a secret or a config map as the build cache node config file, it must be writable.

It is possible to mount secrets or config maps as read-write, but this requires disabling a feature gate and, as such, will be global, which is inadvisable.

If you’re trying to use an immutable source for the configuration, one solution is to mount or write it to a temporary file and then copy that file to config.yaml. However, config updates from the build cache node are not written back to your config source. In this article, you can find an example with more details of the config.yaml file.
 
version: 5
# Restrict UI access
uiAccess:
    type: "secure"
    username: ""
    password: ""
# List users to read/write cache
cache:
    accessControl:
    anonymousLevel: "read"
    users:
        : # e.g developer, reader etc.
            password: ""
            level: "readwrite"
            note: "Developer to read/write cache"
 

For salted hashed password generation, we used this command:

docker run --interactive --tty gradle/build-cache-node:19.0 hashFor this config, we made a Kubernetes secret:

kubectl create secret generic gradle-build-cache-config-secret -n build-cache --from-file=config.yaml

Accessing the build cache node

For accessing the cache server, find the external IP of your Ingress:

kubectl get ingress -n build-cache

Then, you can simply navigate to it from your browser: http://<external-ip>:5071

Configure your Gradle project with the cache node

You can use this snippet as an example of how to configure: settings.gradle
 
 ...
buildCache {
    local {
        enabled = false
    }
    remote(HttpBuildCache) {
        url = 'http://<external IP of your ingress>/cache'
        push = true // Enable pushing to the remote cache
        allowInsecureProtocol = true
        credentials {
            username = 
            password = 
        }
    }
}

Alternative storage solutions

While we chose Azure file share for our implementation, there are a number of alternative storage solutions that might be suitable for different environments and or requirements. For example, Amazon Elastic File System if your team uses AWS and Google Cloud Filestore for GKE as well as many cloud provider agnostic options like Rook, Longhorn, OpenEBS, and so on.

When choosing an alternative, consider the following factors to suit your cloud provider or on-premise infrastructure:

  • Performance requirements.
  • Scalability needs.
  • Data replication and backup features.
  • Cost.
  • Ease of management and integration with existing tools.

For our project, Azure file share provided the right balance of simplicity, performance, and integration with our Azure-based infrastructure. Implementing a Gradle build cache node in AKS significantly improved our build times and overall CI/CD efficiency. However, each solution has its strengths, and the best choice will ultimately depend on your specific use case and environment.

By leveraging Kubernetes StatefulSets and Azure's robust cloud infrastructure, we were able to create a scalable, persistent, and secure solution for our build-cache needs.

I hope this blog post serves as a useful reference for your team to optimize Gradle builds in a Kubernetes environment. Feel free to check out the full implementation details in our GitHub repository.

Published: Sep 19, 2024

Software developmentDevOpsCloud native