Monday, July 25, 2011

Basic Cluster Configuration In JBoss

Share it Please

In this article we will see the basics of Cluster and how we can construct a basic cluster in JBoss. This was worked on RHEL5.

Basically clustering allows us to run applications on several parallel servers while providing a single view to the application client OR a cluster is a set of nodes that communicate with each other and work together towards a common goal.

JBoss provides clustering support in the /server/all server profile. So for this article we will use the all profile for configuring the cluster. Generally in production, there will be apache web server that sends the https requests to the JBoss cluster behind it.
In this article we will see how can connect 2 JBoss servers to form a cluster and verify that they formed a cluster.

A Cluster provides these functionalities:
  • Scalability (can we handle more users? can we add hardware to our system?)
  • Load Balancing (share the load between servers)
  • High Availability (our application has to be uptime close to 100%)
  • Fault Tolerance (High Availability and Reliability)

Initial Preparation
1. Install JBoss Server
2. For each node (server), determine the address (IP) to bind.
3. Ensure multi cast is working. By default JBoss comes with UDP multicast for most intra cluster communication.
4. Unique name for the cluster
5. Unique multi cast address for the cluster
6. Multi cast port.
7. Unique integer "ServerPeerID" for each node
8. Start the cluster.
9. Verify Cluster
10. Basic Issues.

Let’s go step by step

1. Install JBoss Server

We can download JBoss from the Here.

2. For each node (server), determine the address (IP) to bind.

A unique IP address for each node to start.Localhost ( will be sufficient for starting a single server, but will not be useful when we need to work with cluster. So a unique IP address needs to be configured for every server that gets added to the cluster.

3. Ensure multi cast is working. By default JBoss comes with UDP multicast for most intra cluster communication.

A multicast address, also called a group address, is a single IP address for a set of hosts that are joined in a multicasting group.multi cast is nothing but delivery of message to a group of hosts on a single transmission.

So In a JBoss cluster, the servers must communicate with each other to send and receive messages. The communication is done using Jgroups channel. Jgroup library provides the communication support between the servers in the Cluster. It provides a channel on which the communication occurs. This channel handles different tasks such as managing the nodes available in the cluster, detecting failure nodes, identifying new nodes e.t.c.Jgroups also allows to maintain state that is replicated across cluster. Http Session in a web server is replicated to all the servers in the cluster, so that if one node fails the user can log in by the session that is already available.

So when an update is done to a session, the variables that are updated are serialized and send to all other server and the session is updated.

So In order for the JBoss cluster to work, we need to provide a multi cast address.

4. Unique name for the cluster.

A unique name should be assigned to our cluster, JBoss servers that started with the same cluster name has the ability to automatically identify servers that are started with the same cluster name and joins with them to form a cluster.

The default name will be DefaultPartition.

5. Unique multi cast address for the cluster.

JBoss server’s uses UDP multi cast for intra cluster configuration. A multi cast address is selected for the servers to run. Generally a good multicast address is of the form 239.255.x.y.

6. Multi cast port.

Select a unique port for configuring multi cast port. A multi cast port can be add like –m 2000

7. Unique integer "ServerPeerID" for each node

This ServerPeerID is normally used with JBoss messaging. So when you are using Messaging in your cluster, then every server that starts needs to be provided with a ServerPeerID (a Unique Integer ID) which will be consistent even when server restarts.

8. Start the cluster.

So there are two ways to construct a cluster using JBoss servers,

Using a single IP and a single profile (all) and start those using different ports.

In this case, we will use a single IP, a single profile but we will start the servers on different ports. Consider we have a profile all and local host ( now we will start first all profile on local host with port 8080 and second all with another port say 8180.In this way we can make sure that the servers are started on different ports and they don’t collide.

So start the first profile ‘all’ using –c all –Djboss.service.binding.set=ports-01  -g DefaultPartition -b -u

(Access the web console using http://localhost:8180/admin-console)

jboss.service.binding.set: allows us to start the JBoss server with ports that we specify.ports-01 means that ports will start from 8180.This means that a value 100 will be added to the default web server which has a port number 8080 and all other services.

DefaultPartition: a Unique cluster name (-g) a unique IP address (-b) multi cast address (-u)

So let’s start the second profile on the same IP address but with different ports, –c all –Djboss.service.binding.set=ports-02  -g DefaultPartition -b -u

(Check the ports-02, they start with 8280 .Access the web console using http://localhost:8280/admin-console).

Note: In this case, iam not sure whether the servers start correctly or not .Most of the times when I log in to one console, the other console log me out. This may be due to the single IP address we are using).

Using a two IP address and two profiles and start those using different ports.

The better was to use 2 IP address. In this case, we use 2 IP addresses, 2 profiles (a copy of all profile with different name) and start those servers with different ports. So now we will start first all profile on one IP address with port 8080 and second all profile on another IP address with 8180.

We need to use different ports in both the cases, since they may collide if they started on same ports.

First Host IP:
Second Host IP:

Note: For this Demo purpose, I have configured second IP on the same NIC card. Linux has the ability to configure multiple IPs on the same NIC card. How can we configure second IP address on the single NIC card?

So basically when I type ifconfig eth0 on my linux box, I see

eth0              Link encap:Ethernet  HWaddr 00:21:9B:F3:48:4D 
          inet addr:  Bcast:  Mask:
                    inet6 addr: fe80::221:9bff:fef3:484d/64 Scope:Link
                    UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
So I have an IP already configured. So in order to configure second ip address on the same NIC card, we need to edit some configuration files which are located in /etc/sysconfig/network-scripts/

The file ifcfg-eth0 says information about the first eth0 IP information. So now edit ifcfg-eth1 available in the same location and add the following lines,

# please read /usr/share/doc/initscripts-*/sysconfig.txt
# For the documentation of these parameters.

Just add the IP address, netmask and Gateway. Make sure the Device to eth0:1.Once the configuration file is updated. Save it and restart the network using

 service network restart

Once the restart is done, check ifconfig to see 2 IP’s configured.

So the next step is to copy all with a different name all1, so now I have all and all1.

Start the server now (all profile on IP address with port-01, Console URL: http:// -c all -Djboss.service.binding.set=ports-01 -g DefaultPartition -b -u

We can add the multi cast port if we need with –m 2000(any one)
Start the second profile (all1 profile on IP address with port-02, Console URL :
http:// -c all1 -Djboss.service.binding.set=ports-02 -g DefaultPartition -b -u

You can add -Djboss.messaging.ServerPeerID to the above run commands (if you are using messaging).The first all profile should be given -Djboss.messaging.ServerPeerID=1 and second one with -Djboss.messaging.ServerPeerID=2.

9. Verify Cluster

So now we have started the servers, we need to check whether they actually formed a cluster. In order to do this, check the server logs

For the all profile go to /server/all/log/ and search for the text in server.log file.

GMS: address is (cluster=MessagingPostOffice-CTRL)
2011-07-21 11:15:57,476 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) org.jboss.messaging.core.impl.postoffice.GroupMember$ControlMembershipListener@1c18b6a got new view [|1] [,], old view is null
2011-07-21 11:15:57,477 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) I am (
2011-07-21 11:15:57,477 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) New Members : 2 ([,])
2011-07-21 11:15:57,477 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) All Members : 2 ([,])
2011-07-21 11:15:57,556 INFO  [STDOUT] (main)

You can also find similar information in the second all1 profile too.

By this we can make sure that the cluster is configured with 2 servers from IP address and

Another way is to check the servers that added to the cluster from the jmx console. For every server there will be a jmx console which can be accessed using the URL http://

Search for the jboss.jgroups on the left side and click it. Once the right side views are opened search for the GMS bean and click it. The view in the looks like this 

Check the Members attribute, it shows 2 IP address that we started the servers.

10. Basic Issues.

Give a few minutes of time between the starting of the 2 servers.

So in JBoss when we start a server with the cluster information first, it will start all the services and becomes a coordinator and when the second server is started it connects to the coordinator to become a cluster. If we don’t provide enough time between starting the servers, both servers may start at a same time and both may be come coordinator. After some time they both form a cluster and only one server will be a coordinator stopping the services in the other server. Stopping a service in the middle of starting does not always go well. So it is always better to start the server with some time gap between them (say until the first one is started completely)

java.lang.IllegalArgumentException: Cannot start post office since there is already a post office in the cluster with the same node id (0). Are you sure you have given each node a unique node id during installation?

This may be caused due to using the same PeerID for messaging. If you see this error, try to start the servers adding
-Djboss.messaging.ServerPeerID=1 to the first server and -Djboss.messaging.ServerPeerID=2 to the second one.
Modify /etc/hosts file with correct IP information.

Cluster is not connected even with correct details
This may be case when your firewall does not allow multicasting. In RHEL , it may be the work of iptables.
Stop iptables by

service iptables stop

And start the servers one more time.

By this we are now aware of how to construct a JBoss cluster and its necessary details.

More Articles to come, Happy Coding... J

No comments :

Post a Comment