Piotr Woloszyn

World awaits Obama’s magic but I’ll write about something slightly more down-to-earth. I’ve been thinking recently about an implementation possibilities for the dynamic cluster configuration. Lets simplify and assume that a system will consist of identical nodes which can take on different responsibilities depending only on a configuration on the node. The communication between nodes would be achieved by using some kind of messaging technology. So, where am I going with this? It would be good to be able to dynamically reconfigure the cluster nodes to serve as required at the time. The cluster can obviously change over time, some nodes might fail, some other nodes might join the cluster and having a cluster manager which can apply configuration to the current cluster shape and size would be great from flexibility and maintainability perspective. And then the Terracotta comes into the picture. Its feature which can be really useful here is set of cluster events which one can listen to:

• com.tc.cluster.event.thisNodeConnected — The local client connecting to a Terracotta server
• com.tc.cluster.event.thisNodeDisconnected — The local client disconnecting from a Terracotta server
• com.tc.cluster.event.nodeConnected — Another client connecting to a Terracotta server
• com.tc.cluster.event.nodeDisconnected — Another client disconnecting from a Terracotta server

To consume the events we will use javax.management classes and first we will get reference to a MBeanServer:

MBeanServer mbeanServer = ManagementFactory.getPlatformMBeanServer();

Then we should get the cluster bean. I created small helper class ClusterUtils as described in the Terracotta JMX Guide:

ObjectName clusterBean = ClusterUtils.getClusterBean(mbeanServer);

We can alrady check what is the identificator of the current node:

String nodeId = mbeanServer.getAttribute(clusterBean, "NodeId").toString();

The only remaining step is registration to the JMX notifications:

mbeanServer.addNotificationListener(clusterBean, new SimpleJmxNotificationListener(), null, null);

Where the SimpleJmxNotificationListener implements javax.management.NotificationListener. We are ready now to consume the events and react accordingly to the changes in the size of the cluster. There is also a JMX Util library that makes using Terracotta Cluster Events even more easy.

By default the cluster nodes will be indentified by strings in format ClientID[SEQ_NUMBER] like ClientID[85]. If we wanted to set different identification in theory we could use -Dtc.node-name=NAME property when starting a given JVM. However there is currently (Terracotta 2.7.1) an open problem with handling of that property which hopefully will be resolved soon.

Everything what is needed to create a dynamic cluster manager is in place now. Cluster manager would be deployed on each node and share own state in Terracotta cluster. It is not difficult to imagine creating a bit of logic to handle connections/disconnections of new nodes in a cluster and dynamically reconfigure the existing nodes to accommodate to the change. There is always a recurring topic of partitioning when using Terracotta, how to follow paradigm of locality of reference when building a low latency, scalable systems. Combination of the cluster management described above with dynamic subscription to messaging topics with sticky routing could be a nice solution to the problem. Just musing…

Popularity: 45% [?]

Share This