Skip to content
NativeLink Docs

Kubernetes example

This deployment sets up a 4-container deployment with separate CAS, scheduler and worker. Don’t use this example deployment in production. It’s insecure.

In this example we’re using kind to set up the cluster cilium to provide a LoadBalancer and GatewayController.

First set up a local development cluster:

Terminal window
native up

Next start a few standard deployments. This part also builds the remote execution containers and makes them available to the cluster:

Terminal window
./01_operations.sh

Finally, deploy NativeLink:

Terminal window
./02_application.sh

This demo setup creates two gateways to expose the cas and scheduler deployments via your local docker network:

Terminal window
CACHE=$(kubectl get gtw cache-gateway -o=jsonpath='{.status.addresses[0].value}')
SCHEDULER=$(kubectl get gtw scheduler-gateway -o=jsonpath='{.status.addresses[0].value}')


echo "Cache IP: $CACHE"
echo "Scheduler IP: $SCHEDULER"


# Prints something like:
#
# Cache IP: 172.20.255.4
# Scheduler IP: 172.20.255.5

You can now pass these IP addresses to your Bazel invocation to use the remote cache and executor:

Terminal window
bazel build \
    --config=lre \
    --remote_instance_name=main \
    --remote_cache=grpc://$CACHE \
    --remote_executor=grpc://$SCHEDULER \
    //local-remote-execution/examples:hello_lre

When you’re done testing, delete the cluster:

Terminal window
kind delete cluster

Use a published image

Published images can be found under the Container registry, which uses the namespace https://ghcr.io. When using the Container registry, you can select prebuilt images and avoid building the image yourself.

To pull an existing image, you can run:

Terminal window
docker pull ghcr.io/tracemachina/nativelink:taggedImageVersion

Derive a Tag for an OCI image

To derive the tag of the NativeLink image at a specific commit, run the below command and change someCommit with the commit hash you want to use:

Terminal window
nix eval github:TraceMachina/nativelink/someCommit#image.imageTag --raw

Alternatively, the tag can be derived from the upstream sources at the current state of the upstream main branch by running this command:

Terminal window
nix eval github:TraceMachina/nativelink#image.imageTag --raw

Similarly, you can also clone or checkout a specific version or commit of the NativeLink git repository to evaluate the output of the entire NativeLink flake. For example, assuming you’ve done the NativeLink Getting Started Guide and cloned the repository, you can run these sample commands:

Terminal window
git log
git checkout commitHash
nix eval .#image.imageTag --raw

The --raw removes the surrounding quotes from the output string.

Build and copy an OCI image

You can build and copy the image to a container registry with the copyTo attribute. Below are examples within the NativeLink repository for building and copying an image:

Terminal window
nix run github:tracemachina/nativelink#image.copyTo <your destination>
  • Example 2 highlights how to skip pushing to an intermediary registry by copying directly to the docker-daemon:
Terminal window
IMAGE_NAME=$(nix eval .#image.imageName --raw)
IMAGE_TAG=$(nix eval .#image.imageTag --raw)


nix run .#image.copyTo docker-daemon:"${IMAGE_NAME}":"${IMAGE_TAG}"

You can find more about details around nix. Published images are signed using cosign. For more details of the verification process of publishing OCI images see SECURITY.md

Reach out to the NativeLink Slack community for any questions via #NativeLink!