Jakarta EE & Wildfly Running on Kubernetes

In this blog I will explain how to setup and customize Wildfly to run your Jakarta EE application on Kubernetes. We use this setup in our own open source project to run modern Jakarata EE applications on Kubernetes. You can find this project on Github. 

Wildfly is Jakarta EE 8 compatible and includes the latest Eclipse MicroProfile in version 3.3. It provides a modern application framework out of the box to simplify the development of web applications and microservices. All runtime services minimize the heap allocation and applications are starting very fast with a minimum of memory. 

Wildfly 20 – Official Docker Image

Wildfly provides an official container image which is well documented and provides the latest Wildfly containers in version 20. To start Wildfly in a Docker container your can run:

$ docker run -it jboss/wildfly

The Docker image is based with OpenJDK 11 which is optimized to run in container environments like Kubernetes. There are various ways documented on Docker Hub how you can build you own images. In the following section I will show you how we did this in our project.

Build Your Custom Docker Image

To build a Wildfly 20 Docker Container with your own Jakarta EE application you can setup a new Dockerfile and simply copy your application into the deployment directory:

FROM jboss/wildfly
ADD your-awesome-app.war /opt/jboss/wildfly/standalone/deployments/

Wildfly will automatically deploy your application each time your server is starting.

Custom Configuration

In some cases you need to add additional configuration like security realms or database connection pools to run your application. In this case you can customize the main configuration file ‘standalone.xml’ or add additional modules like JDBC drivers. In the following section I will show how to add a PostgresJDBC driver to your Wildfly Server. The separation of external resources like Database Connections from your application is one of the strength in Jakarta EE. In compare to Spring you never touch your exiting application. 

The following directory tree shows a custom configuration adding a PostgreSQL JDBC driver and a custom standalone.xml file

wildfly
    ├── modules
    │   ├── org
    │   │   └── postgresql
    │   │       └── main
    │   │           ├── module.xml
    │   │           └── postgresql-42.2.5.jar
    └── standalone.xml

The JDBC driver is added as a module.  In Imixs-Workflow we use also EclipseLink as the JPA OR-Mapper which is the default mapper in Payara Server. You can add the EclipseLink as a module in a similar way.  You can find the full configuration on Github.

The standalone.xml

To configure the JDBC driver module and a JDBC database pool you can now add the following configuration into the section <datasources> in the standalone.xml. The origin standalone.xml file can be simply copied from a running Wildfly container.

.....
...
<datasource jta="true" jndi-name="java:/jdbc/office"
    pool-name="office" enabled="true" use-ccm="true">
    <connection-url>${env.POSTGRES_CONNECTION}</connection-url>
    <driver-class>org.postgresql.Driver</driver-class>
    <driver>postgresql</driver>
    <security>
        <user-name>${env.POSTGRES_USER}</user-name>
        <password>${env.POSTGRES_PASSWORD}</password>
    </security>
</datasource>
<drivers>
    <driver name="h2" module="com.h2database.h2">
        <xa-datasource-class>org.h2.jdbcx.JdbcDataSource
        </xa-datasource-class>
    </driver>
    <driver name="postgresql" module="org.postgresql">
        <driver-class>org.postgresql.Driver</driver-class>
    </driver>
</drivers>
...
.....

This additional configuration will add the new postgreSQL driver module and a generic database pool which we can configure later in kubernetes by using environment variables. 

The Dockerfile

Now you can add the new configuration files into your Dockerfile and build your new customized Docker image:

FROM jboss/wildfly:20.0.1.Final

COPY ./wildfly/modules/ /opt/jboss/wildfly/modules/
COPY ./wildfly/standalone.xml /opt/jboss/wildfly/standalone/configuration/

# Deploy artefact
ADD your-awesome-app.war /opt/jboss/wildfly/standalone/deployments/

Note: In case you provide custom configuration make sure that you Docker image is based on a specific Wildfly version matching your standalone.xml and module configuration. Otherwise it can happen that a newer Wildfly container will not start with your configuration!

Running in Kubernetes

Now lets run the container in Kubernetes. I assume that you have already an Kubernetes environment. If not take a look into the Open Source project Imixs-Cloud which provides an easy way to setup a self managed Kubernetes cluster. 

The deployment yaml file for the custom Wildfly image can look like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: awesome-app
  labels: 
    app: awesome-app
spec:
  replicas: 1
  selector: 
    matchLabels:
      app: awesome-app
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: awesome-app
    spec:
      containers:
      - name: awesome-app
        image: registry.foo.com/library/awesome-app:latest
        env:
        - name: POSTGRES_CONNECTION
          value: jdbc:postgresql://postgre/my-db
        - name: POSTGRES_PASSWORD
          value: mypassword
        - name: POSTGRES_USER
          value: myuser
        ports: 
          - name: web
            containerPort: 8080
        resources:
          requests:
            memory: "512Mi"
          limits:
            memory: "1Gi"
      restartPolicy: Always

Note that we configure the database pool via the environment variables POSTGRES_CONNECTION, POSTGRES_PASSWORD and POSTGRES_USER as defined before the the standard.xml file from wildfly.  This means our container is still independent form a specific database instance and can be connected to different environments. 

Memory Limits

As mentioned before, OpenJDK11 is optimized to run in containers. With the memory limits defined in the deployment object you can define how the container memory should be managed by Kubernetes:

        resources:
          requests:
            memory: "512Mi"
          limits:
            memory: "1Gi"

In this example we define that our container should start with 512MB and can use a maximum of 1GB. The heap size is automatically controlled by OpenJDK. In most cases there is no additional configuration needed. If for example you want to configure the ratio how OpenJDK uses the available memory for heap and the other stuff you can  provide optional JVM Options:

- name: JAVA_OPTS
  value: "-XX:MaxRAMPercentage=75.0"

Wildfly will automatically use this new Java options on startup. You can easily verify those settings in the containers log file. Depending on your application, additional options may be useful here.

Liveness Probe and Health

With the Eclipse MicroProfile Wildfly provides various metrics and a Health API out of the box. The Health API can be used for a Liveness Probe in Kubernetes.  The MicroProfile Health Check specifications defines three HTTP endpoints: 

• * /health to test both the liveness and readiness of the runtime. 
• * /health/live to test the liveness of the runtime.
•  * /health/ready to test the readiness of the runtime. 

The Health HTTP endpoints are accessible on WildFly HTTP management interface port 9990. To activate the management interface you need to change the default command to start Wildfly at the end of your Dockerfile:

# Run with management interface
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-b", "0.0.0.0", "-bmanagement", "0.0.0.0"]

This will publish port 9990. With the following additional configuration and the corresponding service configuration you can activate the livenessProbe in Kubernetes deploymen yaml files:

        livenessProbe:
          httpGet:
            path: /health
            port: 9990
          initialDelaySeconds: 60
          periodSeconds: 5

This will advice Kubernetes to start checking the Wildfly server every 5 seconds after a start delay of 60 seconds which is sufficient for a complete startup. 

Note: With the Microprofile Health API you can add your individual application specific health checks and use those also for your custom livenessProbe. For example you can implement a check to verify if the database if up and running and is in a consistent  state.

Metrics

By activating the management interface Wildfly also provides you with a large number of server metrics based on the Microprofile Metric API. These metrics can be monitored with Prometheus and Grafana and of course you can add easily your own individual application metrics too!

http://your-server:9990/metrics

The Wildfly Web Console

Wildfly provides a convenient web interface to monitor and manage a running instance.  This interface is accessible via port 9990 but it is protected by default. To activate this interface you can ssh into a running Wildfly container and activate the admin user with the following command:

$ ./wildfly/bin/add-user.sh

You will be prompted to add a new management user with a password. After doing this you can login to the web interface:

http://your-server:9990

Conclusion

Wildfly is ready to run on container based environments like Kubernetes. With the latest Docker image and the Eclipse MicroProfile support you can configure Wildfly in various ways and integrate it smoothly into a microservice architecture. In this short tutorial, I only mentioned the Standalone version of Wildfly. Of course, Wildfly can also be run in a cluster (Domain version) with several instances operating in parallel. The configuration differs only in some minor details.