Build a Container Image Inside a K8s Cluster

Learn how to build a source into a container image from a Dockerfile inside a Kubernetes cluster and push the image to IBM Cloud Container Registry; all of this using Google’s Kaniko tool.

So, What is Kaniko?

Kaniko is a tool to build container images from a Dockerfile, inside a container or Kubernetes cluster.

If you don’t have a Kubernetes cluster with Knative and Istio installed, it’s recommended to follow the instructions mentioned in my previous post that also introduces you to the components of Knative —  Install Knative with Istio and deploy an app on IBM Cloud

apiVersion: build.knative.dev/v1alpha1
kind: BuildTemplate
metadata:
  name: kaniko
spec:
  parameters:
  - name: IMAGE
    description: registry.<region>.bluemix.net/<namespace>/knative-node-kaniko #replace <region> and <namespace>
  - name: DOCKERFILE
    description: ./Dockerfile
    default: /workspace/Dockerfile

  steps:
  - name: build-and-push
    image: gcr.io/kaniko-project/executor
    args:
    - --dockerfile=${DOCKERFILE}
    - --destination=${IMAGE}

•  IMAGE : The Docker image name to apply to the newly built image. Replace <region>and <namespace> with appropriate values. Remember these values as you have to replace these values in the YAMLscripts below.
•  DOCKERFILE : The path to the Dockerfile to execute (default:./Dockerfile).

Note: To check your region, run ibmcloud cr regionand to setup a new namespace, refer this link

If you are looking for a sample with Dockerfile, YAML templates and scripts, Clone this repository

git clone  https://github.com/VidyasagarMSC/knative-deploy

and refer the Kaniko folder. 

Kaniko builds an image and pushes it to the destination defined as a parameter. In order to properly authenticate to the remote container registry (IBM Cloud Container Registry), the build needs to have the proper credentials. This is achieved using a build ServiceAccount.

Before this, let’s define a Secret containing the username and password that the build should use to authenticate (basic) to IBM Cloud Container Registry:

apiVersion: v1
kind: Secret
metadata:
  name: basic-user-pass
  annotations:
    build.knative.dev/docker-0: registry.<region>.bluemix.net # replace the <region>
type: kubernetes.io/basic-auth
stringData:
  username: token # username
  password: <password> # token-value

For <password>, run the below command

$ ibmcloud cr token-add --description “This is a token” --non-expiring --readwrite

Returns: 
Token identifier 58669dd6–3ddd-5c78–99f9-ad0a5aabd9ad 
Token            <token_value>

Use the returned token_value as your password and save the file as secret.yaml . For more details related to token, refer this link

Now you can create aserviceaccount.yaml file with the ServiceAccountusing the secret as shown below

apiVersion: v1
kind: ServiceAccount
metadata:
  name: build-bot
secrets:
- name: basic-user-pass

Let’s use ServiceAccount in our Build and save the file asbuild.yaml

apiVersion: build.knative.dev/v1alpha1
kind: Build
metadata:
  name: kaniko-build
spec:
  serviceAccountName: build-bot
  source:
    git:
      url: https://github.com/VidyasagarMSC/knative-deploy # source code from GitHub
      revision: master
  template:
    name: kaniko
    arguments:
    - name: IMAGE
      value: registry.<region>.bluemix.net/<namespace>/knative-node-kaniko # replace <region> and <namespace>

Execute the build

$ kubectl apply --filename kaniko.yaml
$ kubectl apply --filename secret.yaml
$ kubectl apply --filename serviceaccount.yaml
$ kubectl apply --filename build.yaml

The build should have been kicked off. Let’s take a look.

Runningkubectl get pods , you should see a pod named kaniko-build with a postfix (say XXXXX).

For logs, run this command

$ kubectl logs kanika-build-XXXXX -c build-step-build-and-push

If everything runs as expected, you should see the image in the list when you run the below command

$ ibmcloud cr images

Hurray!! you have just created a container image without a Docker Daemon. Let’s deploy and serve the app so that we can access it from anywhere. For this, lets create a service.yaml file:

apiVersion: serving.knative.dev/v1alpha1 # Current version of Knative
kind: Service
metadata:
  name: knative-node-kaniko # The name of the app
  namespace: default # The namespace the app will use
spec:
  runLatest:
    configuration:
      revisionTemplate:
        spec:
          container:
            image: registry.<region>.bluemix.net/<namespace>/knative-node-kaniko # The URL to the image of the app on IBMCLOUD Registry
            env:
            - name: TARGET # The environment variable printed out by the sample app
              value: "Kaniko Node App running on IBM Cloud"

Execute the service:

$ kubectl apply --filename service.yaml

To find the IP address for your service, use kubectl get svc knative-ingressgateway -n istio-system to get the ingress IP for your cluster. If your cluster is new, it may take some time for the service to get assigned an external IP address.

$ export IP_ADDRESS=$(kubectl get svc knative-ingressgateway --namespace istio-system --output 'jsonpath={.status.loadBalancer.ingress[0].ip}')

To find the URL for your service, use

kubectl get services.serving.knative.dev knative-node-app --output jsonpath='{.status.domain}'

$ export HOST_URL=$(kubectl get services.serving.knative.dev knative-node-kaniko  --output jsonpath='{.status.domain}')

Now you can make a request to your app to see the result.

$ curl -H "Host: ${HOST_URL}" http://${IP_ADDRESS}
Response: Kaniko Node App running on IBM Cloud

$ kubectl delete --filename service.yaml

$ kubectl delete --filename build.yaml
$ kubectl delete --filename serviceaccount.yaml
$ kubectl delete --filename secret.yaml
$ kubectl delete --filename kaniko.yaml

$ ibmcloud cs cluster-rm $CLUSTER_NAME