Skaffold: K8s Development Made Easy

gowebing 2024-12-04

Skaffold is a command line tool developed by Google, which aims to facilitate continuous development for Kubernetes applications. It will automate the task of building and deploying to a Kubernetes cluster whereas you, as a developer, can stay focused on writing code. It seems interesting enough to take a closer look at it!

Introduction

In November 2019 a generally available release was announced promising to save developers time by automating the development workflow. What will Skaffold do for us?

It detects source changes while you are developing
It automatically builds and creates your artifacts (your Docker image) based on a Dockerfile or Jib
It tags the artifacts
It pushes and deploys the artifacts to your Kubernetes cluster

In order to get acquainted with Skaffold, we will run a local Kubernetes cluster with the help of minikube and we will deploy by means of kubectl, the command line interface tool for Kubernetes.

It is advised to take a look at the official Skaffold documentation and examples for more in-depth information. The sources we are using in this post are available at GitHub.

Prerequisites

Before getting started, we need to install minikube, kubectl, and Skaffold if you have not done so already. We will be using Ubuntu 18.04.

Install Minikube

Installation of minikube (version 1.6.2) is quite easy when working with Linux. If you are working with Windows, please take a look at one of our previous posts, it was quite of a struggle way then, but maybe things have improved in the meanwhile.

First, check whether our system has virtualization support enabled:

     Shell 
   
xxxxxxxxxx
 
$ egrep -q 'vmx|svm' /proc/cpuinfo && echo yes || echo no 
yes

The output of the command is yes, which means we do not need to execute any extra steps.

Download and install minikube:

     Shell 
   
xxxxxxxxxx
 
$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_1.6.2.deb && sudo dpkg -i minikube_1.6.2.deb

Start minikube:

     Shell 
   
xxxxxxxxxx
 
$ minikube start
minikube v1.6.2 on Ubuntu 18.04
Automatically selected the 'virtualbox' driver (alternates: [none])
Downloading VM boot image ...
> minikube-v1.6.0.iso.sha256: 65 B / 65 B [--------------] 100.00% ? p/s 0s
> minikube-v1.6.0.iso: 150.93 MiB / 150.93 MiB [-] 100.00% 8.44 MiB p/s 18s
Creating virtualbox VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
Preparing Kubernetes v1.17.0 on Docker '19.03.5' ...
Downloading kubelet v1.17.0
Downloading kubeadm v1.17.0
Pulling images ...
Launching Kubernetes ... 
Waiting for cluster to come online ...
Done! kubectl is now configured to use "minikube"

Install Kubectl

Installation instructions for kubectl can be found here. For Linux, we have to execute the steps below, with kubectl version we verify the successful installation:

     Shell 
   
xxxxxxxxxx
 
$ curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
$ chmod +x ./kubectl
$ sudo mv ./kubectl /usr/local/bin/kubectl
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-07T21:20:10Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-07T21:12:17Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}

Install Skaffold

Installation instructions for Skaffold can be found here. The instructions are very similar to the one for kubectl.

     Shell 
   
xxxxxxxxxx
 
$ curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64
$ chmod +x skaffold
$ sudo mv skaffold /usr/local/bin
$ skaffold version
v1.1.0

Create Demo Application

We will create a simple Spring Boot demo application. We use Spring MVC and create one REST endpoint, which will return a greeting message.

Our pom contains the following dependencies and plugins:

     XML 
   
xxxxxxxxxx
 
<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>
 
<build>
  <plugins>
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
    </plugin>
  </plugins>
</build>

The HelloController contains one method, which returns the greeting message and the address of the host where it is executed.

     Java 
   
xxxxxxxxxx
 
@RestController
public class HelloController {
 
    @RequestMapping("/hello")
    public String hello() {
        StringBuilder message = new StringBuilder("Hello Skaffold!");
        try {
            InetAddress ip = InetAddress.getLocalHost();
            message.append(" From host: " + ip);
        } catch (UnknownHostException e) {
            e.printStackTrace();
        }
        return message.toString();
    }
 
}

Use Skaffold

Now that we have finished all the necessary preparations, it is time to start using Skaffold. Currently, we deliberately have left out some important configuration, which Skaffold will need, but this will allow us to verify which error messages are returned and how to solve them.

Generate skaffold.yaml

Skaffold will need a skaffold.yaml file that contains the development workflow you want to use. The file can be generated when the init command is executed from within your project directory.

     Shell 
   
xxxxxxxxxx
 
$ skaffold init
FATA[0000] one or more valid Kubernetes manifests is required to run skaffold

Skaffold init does not create the Kubernetes manifest files for us, we will need to create them manually.

We will create a k8s deployment file with the help of kubectl. We copy the output of the command to file deployment.yaml in the k8s directory. The CLI parameter --dry-run ensures us that the deployment is not executed yet, the parameter -oyaml will output the configuration, which will allow us to just copy the contents.

     Shell 
   
xxxxxxxxxx
 
$ kubectl create deployment myskaffoldplanet --image=docker.io/mydeveloperplanet/myskaffoldplanet --dry-run -oyaml
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: myskaffoldplanet
    name: myskaffoldplanet
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myskaffoldplanet
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: myskaffoldplanet
    spec:
      containers:
      - image: docker.io/mydeveloperplanet/myskaffoldplanet
        name: myskaffoldplanet
        resources: {}
status: {}

Running skaffold init again, returns a new error:

     Shell 
   
xxxxxxxxxx
 
$ skaffold init
FATA[0000] one or more valid builder configuration (Dockerfile or Jib configuration) must be present to build images with skaffold; please provide at least one build config and try again or run `skaffold init --skip-build`

We could have expected this, because we did not provide a Dockerfile or Jib configuration. We will make use of Jib after our positive experiences with it in our previous post. Add the Jib Maven Plugin to our pom. We do not provide credentials this time, because we are not going to push the Docker image to a Docker registry.

     XML 
   
xxxxxxxxxx
 
<plugin>
  <groupId>com.google.cloud.tools</groupId>
  <artifactId>jib-maven-plugin</artifactId>
  <version>1.8.0</version>
  <configuration>
    <!-- openjdk:11.0.5-jre -->
    <from>
      <image>openjdk@sha256:b3e19d27caa8249aad6f90c6e987943d03e915bbf3a66bc1b7f994a4fed668f6</image>
    </from>
    <to>
      <image>docker.io/${docker.image.prefix}/${project.artifactId}</image>
      <tags>
        <tag>${project.version}</tag>
      </tags>
    </to>
    <container>
      <mainClass>com.mydeveloperplanet.myskaffoldplanet.MySkaffoldPlanetApplication</mainClass>
      <user>nobody</user>
    </container>
  </configuration>
</plugin>

In order to use Jib, we need to add the flag --XXenableJibInit, see also this issue.

     Shell 
   
xxxxxxxxxx
 
$ skaffold init --XXenableJibInit
apiVersion: skaffold/v2alpha1
kind: Config
metadata:
  name: myskaffoldplanet
build:
  artifacts:
  - image: docker.io/mydeveloperplanet/myskaffoldplanet
    jib: {}
deploy:
  kubectl:
    manifests:
    - k8s/deployment.yaml
 
Do you want to write this configuration to skaffold.yaml? [y/n]: y
Configuration skaffold.yaml was written
You can now run [skaffold build] to build the artifacts
or [skaffold run] to build and deploy
or [skaffold dev] to enter development mode, with auto-redeploy

Skaffold Continuous Development

We have set up all necessary configuration in order to experiment with the skaffold dev command. This will scan our project for any changes and automatically build and deploy them to our Kubernetes cluster. Run the following command:

     Shell 
   
xxxxxxxxxx
 
$ skaffold dev

Our application is being built and deployed to our Kubernetes cluster. We can also verify this with the minikube dashboard:

     Shell 
   
xxxxxxxxxx
 
$ minikube dashboard

We cannot invoke our URL yet because we did not create a service yet. We will map port 8080 via a NodePort. Generate the service yaml and add the contents (without the labels) to file service.yaml in the k8s directory:

     Shell 
   
xxxxxxxxxx
 
$ kubectl expose deployment myskaffoldplanet --type=NodePort --port=8080 --dry-run -oyaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: myskaffoldplanet
    app.kubernetes.io/managed-by: skaffold-v1.1.0
    skaffold.dev/builder: local
    skaffold.dev/cleanup: "true"
    skaffold.dev/deployer: kubectl
    skaffold.dev/docker-api-version: "1.40"
    skaffold.dev/run-id: c8fc23d2-85f5-453a-bc22-19f4a9ec88a6
    skaffold.dev/tag-policy: git-commit
    skaffold.dev/tail: "true"
  name: myskaffoldplanet
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: myskaffoldplanet
  type: NodePort
status:
  loadBalancer: {}

Also, add the service.yaml as manifest file to the skaffold.yaml file:

     YAML 
   
xxxxxxxxxx
 
deploy:
 kubectl:
   manifests:
   - k8s/deployment.yaml
   - k8s/service.yaml

Skaffold immediately notices this change and creates the service. This can be verified in the Skaffold console output:

     Shell 
   
xxxxxxxxxx
 
Starting deploy...
- deployment.apps/myskaffoldplanet configured
- service/myskaffoldplanet created
Watching for changes...

Verify the creation of the service with kubectl:

     Shell 
   
xxxxxxxxxx
 
$ kubectl get services
NAME                 TYPE         CLUSTER-IP     EXTERNAL-IP    PORT(S)           AGE
kubernetes           ClusterIP    10.96.0.1                     443/TCP           24h
myskaffoldplanet     NodePort     10.96.65.87                   8080:30272/TCP    42s

The NodePort that has been assigned is port 30272 (see PORT(S) column). We are now able to invoke our Rest endpoint:

     Shell 
   
xxxxxxxxxx
 
$ curl $(minikube ip):30272/hello
Hello Skaffold! From host: myskaffoldplanet-76f44959c9-tcvw5/172.17.0.6

Change the greeting text in the HelloController:

     Java 
   
xxxxxxxxxx
 
StringBuilder message = new StringBuilder("Hello there, Skaffold!");

Again, the change is automatically detected by Skaffold, and in the background, our application is being built and deployed. Invoke the URL again:

     Shell 
   
xxxxxxxxxx
 
$ curl $(minikube ip):30272/hello
Hello there, Skaffold! From host: myskaffoldplanet-54b59fb785-hczn8/172.17.0.7

We can also use command skaffold run in order to deploy on request:

     Shell 
   
xxxxxxxxxx
 
$ skaffold run
...
Starting deploy...
- deployment.apps/myskaffoldplanet created
- service/myskaffoldplanet created
You can also run [skaffold run --tail] to get the logs

Again, check the NodePort the service is running and invoke the URL.

Troubleshooting

We once encountered the following error when running skaffold dev and skaffold run:

     Shell 
   
xxxxxxxxxx
 
rpc error: code = Unknown desc = Error response from daemon: pull access denied for mydeveloperplanet/myskaffoldplanet, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

We solved this by adding the following lines to the skaffold.yaml file. Later on, it seemed not to be necessary anymore.

     YAML 
   
xxxxxxxxxx
 
build:
 local:
   push: false
 artifacts:
 - image: docker.io/mydeveloperplanet/myskaffoldplanet
   jib: {}

Conclusion

In this post, we looked at Skaffold for automatic building and deploying our application to a Kubernetes cluster during development. We only scratched the surface of what Skaffold is capable of, but we were very impressed by this tool. Definitely something to use and to keep an eye on.