Java High Availability With WildFly on Kubernetes
legacy java applications
with the advancement of the dockerization of applications, kubernetes has become a standard in the market, but we must remember that there are still thousands of legacy applications that depend on certain features provided by the application servers. so, if you need to use session replication, you will surely need wildfly instances to be clustered. to solve this smoothly, you can use the “kubernetes discovery protocol for jgroups” aka “kube-ping”. kube_ping is a discovery protocol for jgroups cluster nodes managed by kubernetes: jgroups-kubernetes .
walkthrough
i assume you already have a kubernetes cluster, so let’s just focus on wildfly settings.
the first step is to create a repository that will contain all our files:
shell
xxxxxxxxxx
1
1
[root@workstation ~]# mkdir wildfly20-kubeping
2
[root@workstation ~]# mkdir -p wildfly20-kubeping/application
3
[root@workstation ~]# mkdir -p wildfly20-kubeping/configuration
4
[root@workstation ~]# mkdir -p wildfly20-kubeping/kubernetes
then create a dockerfile with the necessary steps for image customization:
dockerfile
xxxxxxxxxx
1
16
1
[root@workstation ~]# vi wildfly20-kubeping/dockerfile
2
from jboss/wildfly:20.0.0.final
3
4
label maintainer mauricio magnani <msmagnanijr@gmail.com>
5
6
run /opt/jboss/wildfly/bin/add-user.sh admin redhat --silent
7
8
add configuration/config-server.cli /opt/jboss/
9
10
run /opt/jboss/wildfly/bin/jboss-cli.sh --file=config-server.cli
11
12
run rm -rf /opt/jboss/wildfly/standalone/configuration/standalone_xml_history/*
13
14
add application/cluster.war /opt/jboss/wildfly/standalone/deployments/
15
16
expose 8080 9990 7600 8888
in the same directory, create the file “config-server.cli”, which will contain the steps to configure kubeping correctly:
shell
xxxxxxxxxx
1
21
1
[root@workstation ~]# vi wildfly20-kubeping/configuration/config-server.cli
2
###start the server in admin-only mode, using/modifying standalone-full-ha.xml
3
embed-server --server-config=standalone-full-ha.xml --std-out=echo
4
5
###apply all configuration to the server
6
batch
7
#/subsystem=logging/logger=org.openshift.ping:add()
8
#/subsystem=logging/logger=org.openshift.ping:write-attribute(name=level, value=debug)
9
#/subsystem=logging/logger=org.openshift.ping:add-handler(name=console)
10
/subsystem=messaging-activemq/server=default/cluster-connection=my-cluster:write-attribute(name=reconnect-attempts,value=10)
11
/interface=kubernetes:add(nic=eth0)
12
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp/:write-attribute(name=interface,value=kubernetes)
13
/socket-binding-group=standard-sockets/socket-binding=jgroups-tcp-fd/:write-attribute(name=interface,value=kubernetes)
14
/subsystem=jgroups/channel=ee:write-attribute(name=stack,value=tcp)
15
/subsystem=jgroups/channel=ee:write-attribute(name=cluster,value=kubernetes)
16
/subsystem=jgroups/stack=tcp/protocol=mping:remove()
17
/subsystem=jgroups/stack=tcp/protocol=kubernetes.kube_ping:add(add-index=0,properties={namespace=${env.kubernetes_namespace},labels=${env.kubernetes_labels},port_range=0,masterhost=kubernetes.default.svc,masterport=443})
18
run-batch
19
20
###stop embedded server
21
stop-embedded-server
now put the “package of your application” to the directory “application”
shell
xxxxxxxxxx
1
1
[root@workstation ~]# cp cluster.war wildfly20-kubeping/application
the next step is to create all kubernetes objects so we can do the tests later. i’m using the namespace “labs”:
dockerfile
xxxxxxxxxx
1
122
1
[root@workstation ~]# vi wildfly20-kubeping/kubernetes/01-wildfly-sa-role.yaml
2
3
apiversion: v1
4
kind: serviceaccount
5
metadata:
6
name: jgroups-kubeping-service-account
7
namespace: labs
8
---
9
kind: clusterrole
10
apiversion: rbac.authorization.k8s.io/v1
11
metadata:
12
name: jgroups-kubeping-pod-reader
13
namespace: labs
14
rules:
15
- apigroups: [""]
16
resources: ["pods"]
17
verbs: ["get", "list"]
18
---
19
apiversion: rbac.authorization.k8s.io/v1beta1
20
kind: clusterrolebinding
21
metadata:
22
name: jgroups-kubeping-api-access
23
roleref:
24
apigroup: rbac.authorization.k8s.io
25
kind: clusterrole
26
name: jgroups-kubeping-pod-reader
27
subjects:
28
- kind: serviceaccount
29
name: jgroups-kubeping-service-account
30
namespace: labs
31
32
[root@workstation ~]# vi wildfly20-kubeping/kubernetes/02-wildfly-deployment.yaml
33
34
apiversion: apps/v1
35
kind: deployment
36
metadata:
37
name: wildfly
38
namespace: labs
39
labels:
40
app: wildfly
41
tier: devops
42
spec:
43
selector:
44
matchlabels:
45
app: wildfly
46
tier: devops
47
replicas: 2
48
template:
49
metadata:
50
labels:
51
app: wildfly
52
tier: devops
53
spec:
54
serviceaccountname: jgroups-kubeping-service-account
55
containers:
56
- name: kube-ping
57
image: mmagnani/wildfly20-kubeping:latest
58
command: ["/opt/jboss/wildfly/bin/standalone.sh"]
59
args: ["--server-config", "standalone-full-ha.xml", "-b", $(pod_ip), "-bmanagement", $(pod_ip) ,"-bprivate", $(pod_ip) ]
60
resources:
61
requests:
62
memory: 256mi
63
limits:
64
memory: 512mi
65
imagepullpolicy: always
66
ports:
67
- containerport: 8080
68
- containerport: 9990
69
- containerport: 7600
70
- containerport: 8888
71
env:
72
- name: pod_ip
73
valuefrom:
74
fieldref:
75
apiversion: v1
76
fieldpath: status.podip
77
- name: kubernetes_namespace
78
valuefrom:
79
fieldref:
80
apiversion: v1
81
fieldpath: metadata.namespace
82
- name: kubernetes_labels
83
value: app=wildfly
84
- name: java_opts
85
value: -djdk.tls.client.protocols=tlsv1.2
86
87
[root@workstation ~]# vi wildfly20-kubeping/kubernetes/03-wildfly-service.yaml
88
89
apiversion: v1
90
kind: service
91
metadata:
92
name: wildfly
93
namespace: labs
94
labels:
95
app: wildfly
96
tier: devops
97
spec:
98
type: clusterip
99
ports:
100
- targetport: 8080
101
port: 8080
102
selector:
103
app: wildfly
104
tier: devops
105
106
[root@workstation ~]# vi wildfly20-kubeping/kubernetes/04-wildfly-ingress.yaml
107
108
---
109
apiversion: extensions/v1beta1
110
kind: ingress
111
metadata:
112
name: wildfly-ingress
113
namespace: labs
114
spec:
115
rules:
116
- host: wildfly.mmagnani.lab
117
http:
118
paths:
119
- path: /
120
backend:
121
servicename: wildfly
122
serviceport: 8080
now just run the image build and push this to the docker hub or any other “registry”:
shell
xxxxxxxxxx
1
1
[root@workstation ~]# docker build -t mmagnani/wildfly20-kubeping:latest .
2
[root@workstation ~]# docker push mmagnani/wildfly20-kubeping
in the context of your kubernetes cluster, just create those objects:
shell
xxxxxxxxxx
1
1
[root@k8s-master kubernetes]# kubelet --version
2
kubernetes v1.18.3
3
[root@k8s-master kubernetes]# pwd
4
/root/wildfly20-kubeping/kubernetes
5
[root@k8s-master kubernetes]# kubectl create -f . -n labs
6
[root@k8s-master kubernetes]# kubectl get pods -n labs
7
name ready status restarts age
8
wildfly-cc8b9546f-df9sw 1/1 running 0 48s
9
wildfly-cc8b9546f-scfdp 1/1 running 0 48s
check the logs. you should find two members as we only define 2 replicas:
plain text
xxxxxxxxxx
1
1
[root@k8s-master kubernetes]# kubectl logs -f wildfly-cc8b9546f-df9sw -n labs
2
3
14:23:33,253 info [org.infinispan.cluster] (msc service thread 1-2) ispn000078: starting jgroups channel kubernetes
4
14:23:33,253 info [org.infinispan.cluster] (msc service thread 1-2) ispn000094: received new cluster view for channel kubernetes: [wildfly-cc8b9546f-scfdp|1] (2) [wildfly-cc8b9546f-scfdp, wildfly-cc8b9546f-df9sw]
as i am using “traefik”, my “ingress” is available:
in the application, it is also possible to add the object to the session:
