Piotr Woloszyn

World awaits Obama’s magic but I’ll write about something slightly more down-to-earth. I’ve been thinking recently about an implementation possibilities for the dynamic cluster configuration. Lets simplify and assume that a system will consist of identical nodes which can take on different responsibilities depending only on a configuration on the node. The communication between nodes would be achieved by using some kind of messaging technology. So, where am I going with this? It would be good to be able to dynamically reconfigure the cluster nodes to serve as required at the time. The cluster can obviously change over time, some nodes might fail, some other nodes might join the cluster and having a cluster manager which can apply configuration to the current cluster shape and size would be great from flexibility and maintainability perspective. And then the Terracotta comes into the picture. Its feature which can be really useful here is set of cluster events which one can listen to:

• com.tc.cluster.event.thisNodeConnected — The local client connecting to a Terracotta server
• com.tc.cluster.event.thisNodeDisconnected — The local client disconnecting from a Terracotta server
• com.tc.cluster.event.nodeConnected — Another client connecting to a Terracotta server
• com.tc.cluster.event.nodeDisconnected — Another client disconnecting from a Terracotta server

To consume the events we will use javax.management classes and first we will get reference to a MBeanServer:

MBeanServer mbeanServer = ManagementFactory.getPlatformMBeanServer();

Then we should get the cluster bean. I created small helper class ClusterUtils as described in the Terracotta JMX Guide:

ObjectName clusterBean = ClusterUtils.getClusterBean(mbeanServer);

We can alrady check what is the identificator of the current node:

String nodeId = mbeanServer.getAttribute(clusterBean, "NodeId").toString();

The only remaining step is registration to the JMX notifications:

mbeanServer.addNotificationListener(clusterBean, new SimpleJmxNotificationListener(), null, null);

Where the SimpleJmxNotificationListener implements javax.management.NotificationListener. We are ready now to consume the events and react accordingly to the changes in the size of the cluster. There is also a JMX Util library that makes using Terracotta Cluster Events even more easy.

By default the cluster nodes will be indentified by strings in format ClientID[SEQ_NUMBER] like ClientID[85]. If we wanted to set different identification in theory we could use -Dtc.node-name=NAME property when starting a given JVM. However there is currently (Terracotta 2.7.1) an open problem with handling of that property which hopefully will be resolved soon.

Everything what is needed to create a dynamic cluster manager is in place now. Cluster manager would be deployed on each node and share own state in Terracotta cluster. It is not difficult to imagine creating a bit of logic to handle connections/disconnections of new nodes in a cluster and dynamically reconfigure the existing nodes to accommodate to the change. There is always a recurring topic of partitioning when using Terracotta, how to follow paradigm of locality of reference when building a low latency, scalable systems. Combination of the cluster management described above with dynamic subscription to messaging topics with sticky routing could be a nice solution to the problem. Just musing…

Popularity: 45% [?]

Share This

Apache Qpid is an open source implementation of Advance Message Queuing Protocol (AMQP), an emerging open standard for inter-operable Message Oriented Middleware (MOM) communication in financial market data delivery. I have been looking recently at Red Hat MRG Messaging which is based on Apache Qpid.
The feature of Qpid which I wanted to write about today is called Last Value Queues. Definition from MRG User Guide:

“The last value queue type causes logically updated versions of previous messages to appear to overwrite the older messages. To use this feature, add a qpid.last_value_queue key to the arguments supplied. The value of the key is irrelevant. Messages published to the queue then need to specify a value for the qpid.LVQ_key in the headers of messages they publish. The last value queue uses the value of the qpid.LVQ_key to determine whether a newly published message is an update to an existing message on the queue. If this is the case, the new message will appear to overwrite the older message. A subscriber that requests messages after this has occurred will see only the newer message.”

Common example would be a queue with latest price updates per symbol when we are interested only in the last value per instrument. The documentation of MRG Messaging mentions only last value queues defined by providing qpid.last_value_queue parameter when creating a queue. There is also slightly different implementation of this feature defined by parameter qpid.last_value_queue_no_browse. The difference is described on qpid website.

Let start and create a last value queue using qpid-config tool:

qpid-config add queue lvq --last-value-queue

Check if the queue has been created:

qpid-config queues

Queue Name                                        Attributes
======================================================================
lvq                                               --last-value-queue

The messages with the same value of the qpid.LVQ_key property will be replaced, so if we wanted to write simple price publisher that publishes 12 messages of four symbols SYMBOLS = new String [] {”Crude”, “Gold”, “Silver”, “Copper”} which uses the last value queue feature the code would look like:

Destination destinationQueue = new AMQQueue("amq.direct", "lvq");
MessageProducer producerQueue = session.createProducer(destinationQueue);
TextMessage txtQueueMsg = session.createTextMessage();
for (int i=1; i < = 12; i++) {
    String symbol = SYMBOLS[(i-1)%4];
    txtQueueMsg.setStringProperty("qpid.LVQ_key", symbol);
    txtQueueMsg.setText(String.format("Tick %d for symbol %s", i, symbol));
    producerQueue.send(txtQueueMsg);
    log.info(String.format("Producer: sent message => %s",  txtQueueMsg.getText()));
}

Let’s create a synchronous client to consume the messages:

Destination destinationQueue = new AMQQueue("amq.direct", "lvq");
MessageConsumer consumerQueue = session.createConsumer(destinationQueue);

boolean finished = false;
while(!finished) {
    TextMessage message = (TextMessage) consumerQueue.receive(5000);
    if (message != null) {
        log.info(String.format("Consumer: received message %s key=%s", message.getText(), message.getStringProperty("qpid.LVQ_key")));
        message.acknowledge();
    } else {
        log.info("Timeout");
        finished = true;
    }
}

Assume that consumer subscribed to the queue after publisher finished publishing the 12 messages. In this case the consumer will receive only 4 messages, one per symbol:

Consumer: received message Tick 9 for symbol Crude key=Crude
Consumer: received message Tick 10 for symbol Gold key=Gold
Consumer: received message Tick 11 for symbol Silver key=Silver
Consumer: received message Tick 12 for symbol Copper key=Copper

Popularity: 84% [?]

Share This

Close to two years ago I had my first look at Terracotta when researching available technology for sharing state and scaling Aloha. First impressions were good. Terracotta delivered what promised: easy integration with an existing single node application and virtual shared heap available across whole cluster. However it has much more to offer. I have been looking again recently in the distributed cache and data grids area trying to assess options for a project I am working on currently. Below is a digest from my web-crawling of publicly available data on the topic. In each of the paragraphs I will try to put some light on a specific aspect that someone would like to know when considering Terracotta.

Resilience

Terracotta clusters offer simple and scalable HA implementations based on the Terracotta server array. Terracotta server arrays can vary from a basic two-node array consisting of an active server instance backed by a hot standby (for instant failover), to a multi-node array with many active server instances providing scale and deep failover coverage. Terracotta give high availability of the data by not only replicating it across servers for redundancy, but also by providing a simultaneous permanent storage of all current shared (in-memory) data to disk – available to all server instances (no loss of application state). Temporarily disconnected server instances and clients are automatically reconnected, hot standbys restored without operator intervention. There is no single point of failure and bottleneck in the Terracotta architecture.

Failover

Terracotta server array provides instant failover and continuous uptime and services using a hot standby or multiple active servers. Multiple Terracotta servers can be clustered together in active-active or active-passive configurations, helping to further scale Java applications and to provide disaster recovery.

DB Persistence

Terracotta helps to protect database in a scale out architecture. There is a problem with usual approach with a cache in front of the database. The performance of the cache is degradated when the number of nodes increases. Hibernate offers a second-level cache option, which can improve the performance of the Hibernate application by eliminating the need to fetch often used data objects from the database, instead storing, or caching, that data in process with your application. Without any mechanism to provide replication of the underlying cache, data written in one node can be stale on another node. This problem is not solved by Hibernate, rather it is up to the underlying cache to provide a replication scheme to keep multiple copies synchronized. Terracotta provides a consistent mechanism to keep any cache written in Java synchronized, and therefore provides this support for caches such as EHCache. Advantages: Performance — Reading and writing to replicated data with Terracotta can be 10x - 100x faster than other solutions, Cluster-wide load — Replication of data eliminates the need for more than one node to load data from the database. In a scaled-out application, this can reduce database load by a significant factor due to the fact that only one node needs to request any piece of data for a cache miss and all subsequent requests on other nodes will result in cache hits.

Distributed caching implementation

Terracotta uses concept of the Network-Attached Memory. It clusters JVMs directly underneath applications providing distributed, shared heap. The implementation doesn’t rely on serialization of whole objects, rather on instrumentation of the byte code and injecting hooks in there which allow synchronizing of deltas in the object state. State changes in a Client JVM are published to Terracotta Server Array and all other Client JVMs are seamlessly updated. Other Terracotta’s features apart from shared heap are: distributed locking, distributed data structures – (HashMap, ArrayList, LinkedBlockingQueue, any POJO), guaranteed coherent clustered objects. Objects do not have to implement Serializable to work with Terracotta.

OS/Platform restrictions

Terracotta is a 100% Java solution.

Certified platforms: Solaris 10, Solaris 9, RedHat ES4, SUSE ES10.1 (32/64bit), Windows Server 2003 R2, WindowsXP (dev only)
Server JDKs: Sun Hotspot 1.5.0_16, Sun Hotspot 1.6.0_07
Client JDKs: Sun Hotspot 1.4.2_18, Sun Hotspot 1.5.0_16, Sun Hotspot 1.6.0_07
Containers: Tomcat 6.0.18 / 5.5.26 / 5.0.28, Apache Geronimo 1.1.1, BEA WebLogic 8.1 sp6, BEA WebLogic 9.2.MP2, JBoss 3.2.8.sp1 / 4.0.5, Jetty 6.1.11, WebSphere CE 1.0.1.2 / AS 6.1.0.7, Glassfish V1 ur1-p01-b02, Glassfish-v2 ur2-b04

Ease of implementation

Big advantage of Terracotta. A clustered application looks no different than a standard Java applications. In theory a correct multi-threaded application can be clustered just by providing a simple xml config defining roots of application state hierarchy which one wants to share across multiple JVMs. Distributed locking and thread coordination uses usual language artefacts like ’synchronized’, ‘wait’/'notify’ and java.util.concurrent support. Pre-existing Terracotta Integration Modules (TIMs): purpose-built, tested, and tuned to implement a particular clustered architecture pattern or integrate with a particular third-party technology, for example use TIMs for Spring, Hibernate, Quartz, Compass, Lucene, Guice, Camel, Struts, Joda time and more to seamlessly add clustering to an application. Last but not least: Terracotta is an open source product with an active community.

Hardware requirements

Client:

Memory Sizing - Java Heap: each client JVM should do work on data that fits in its local heap
Memory Sizing - RAM: enough RAM to accommodate the heap of the Terracotta client JVM
Number of client nodes: determined by the total size of the active clustered data set and total required throughput. Entire data set will fit in the sum of the total heap available for clustered object storage across all Terracotta client JVMs.

Server:

Memory Sizing - Java Heap: Ideally, all of the clustered object data should fit into its heap, note: allow for garbaged objects, not recommended heap > 6/8GB
Memory Sizing - RAM: enough RAM to accommodate the heap of the Terracotta server JVM. In addition, there should be enough RAM headroom to keep the blocks of the Sleepycat data store files in the host operating system’s filesystem cache for fast access.
CPU: dedicated server, capacity planning needed. Note: The more garbage your application creates, the more CPU resources will be consumed by the Terracotta Distributed Garbage Collector. Likewise, the higher the object read or write rate between Terracotta client nodes and a Terracotta server instance, the more CPU resources that server instance consumes.
Number of server nodes: min 2 for HA and failover

Network

Gigabit full duplex between server and clients, and server array, low latency network, redundant NICs and redundant switches with automatic failover

Performance

According to Terracotta: it is faster than other distributed cache, data grid solutions. It is due to the nature of synchronizing only deltas of object state. Terracotta scales much better than i.e. Coherence even with big transaction payloads. At small scale it has throughput 2x Oracle’s at all transaction sizes. At high scale (8 nodes) throughput was as much as 10x higher than Oracle. However independent tests of a specific use case scenario should be performed to back up the results.

Support package

There are Enterprise Subscription packages: production support GOLD 7X24 | SILVER 5X12 with issue escalation privileges and procedures. Also consulting and training services. Pricing individually discussed.

Code stability

Current version 2.7.1. Code is mature as the company started to work on Terraccota in 2003. Two years ago project has been opensourced. Licensed as Terracotta Public License, based on the Mozilla Public License. Free version requires displaying the words “Powered by Terracotta” in user interfaces. Commercial Version removes this requirement. Typical price $7,000 per node. A single Terracotta server can handle up to 150 nodes, but on heavy Java applications, such as those used in online reservation systems, that gets reduced to somewhere between 15 to 20 nodes per Terracotta server.

Partitioning

Workload partitioning:

Terracotta does not ship with a particular workload partitioning strategy in the box. Layer 7 sticky load balancer that is able to route requests for a particular user session to the same Terracotta client application server and is likewise able to balance user sessions evenly across all available application servers can be used to achieve workload partitioning. Use of round-robin or a non-sticky load balancing is strongly discouraged. Other forms of workload partitioning may be required for different use cases. Implementing workload partitioning algorithms in Java that can then be made cluster-aware via Terracotta is often a simple way to meet custom workload partitioning requirements.

Data partitioning

Each application server should only have the data that its workload requires access to in heap. (Magic warning!) Terracotta detects how your application uses its data and optimizes the data location automatically to minimize network updates. Terracotta manages replicating in the most efficient way. Only deltas to your object graph are pushed, objects automatically reside in JVMs that need them, and not in JVMs that don’t (vs. alternatives that always force you to configure and partition up front).

Transactional

Terracotta transactions are sets of clustered object changes that must be applied atomically. Transactions are bounded by lock acquisition and release. When a clustered lock is acquired by a thread, a Terracotta transaction is started and all changes to clustered objects made within the scope of that lock are added to that transaction. When the lock is released, the Terracotta transaction is committed. If a thread attempts to modify a clustered object outside the context of a terracotta transaction, a runtime exception will be thrown.

Replication and consistency

Fully synchronous replication between nodes. The data is always coherent and doesn’t suffer any cache drifts.

Popularity: 64% [?]

Share This

It is an example how to use the Web21c SDK from a Scala application.

First couple of words about Scala. Scala was created in 2001 at the École Polytechnique Fédérale de Lausanne (EPFL) in Switzerland. It is platform independent, multi-paradigm language which integrates features of object-oriented and functional programming, is type-safe (but you don’t have to define redundant types of information) and extensible (good candidate for DSLs). It brings a lot of very nice syntactical sugar as well! As Scala smoothly integrates with Java and .Net worlds we can run Scala applications on JVM or .NET CLR and easily reuse existing libraries for those platforms.

One of those libraries is Web21c SDK which gives us easy way of integrating communication services and comes in couple of flavours: SDK for Java, .Net, Python and PHP. We could integrate Scala with Java or .NET SDK but for the purpose of this example we will see how easy it is to incorporate Java SDK into our Scala application.

Let’s assume we use Scala plugin for Eclipse and we created a new Scala project.

We can get the latest Web21c SDK for Java and before proceeding to next steps we have to register our application and obtain own certificate for sandbox server which is free. Then we will create two folders: lib and resources in our project and we will unzip content of the Web21C-JavaSDK-5.0.zip and copy content of the lib folder from SDK to our lib folder. Let’s copy the Web21C-JavaSDK-5.0.jar there as well. We will copy previously obtained certificate to the resources folder and we will create a properties file called security.properties where you can store name of your cert and its password:

org.apache.ws.security.crypto.provider=com.bt.security.PKCS12Crypto
org.apache.ws.security.crypto.merlin.keystore.type=pkcs12
org.apache.ws.security.crypto.merlin.keystore.password=YOUR_CERTIFICATE_PASSWORD
org.apache.ws.security.crypto.merlin.alias.password=YOUR_CERTIFICATE_PASSWORD
org.apache.ws.security.crypto.merlin.file=YOUR_CERTIFICATE.pfx

constants.provider=com.bt.sdk.common.SandboxConstants

Next from Eclipse in the project Properties/Java Build Path/Source/Add Folder… we will add resources folder to the classpath and all jars from lib folder to the Properties/Java Build Path/Libraries/Add JARs… to place them on the classpath as well. We should be good to go.

Then we will create the Scala singleton where we will use directly Java objects. For example Manager classes for one of the Web21c services, for instance One Way Messaging to create a simple sms client (src/SmsClient.scala):

import com.bt.sdk.capabilities.messaging.oneway.MessagingOneWayManager

object SmsClient {
    def main(args: Array[String]) {
        new MessagingOneWayManager().sendMessage("tel:44xxxxxxxxx", "Hello World")
    }
}

Or even simpler:

import com.bt.sdk.capabilities.messaging.oneway.MessagingOneWayManager

object SmsClient extends Application {
    new MessagingOneWayManager().sendMessage("tel:44xxxxxxxxx", "Hello World")
}

Is that close to the famous “one line of code”?

Our project’s structure should look more or less like:

+ Web21cAndScala
    L---+ src
        L--- Web21c.scala
    L---+ bin
        L--- //compiled classes
    L---+ lib
        L--- Web21C-JavaSDK-5.0.jar
        L--- //other jars from lib folder
    L---+ resources
        L--- sandbox.pfx
        L--- security.properties

We are ready to try if it works. Try Run As… Scala Application. Great if it works! If you get following error:

Exception in thread "main" Reporting Service: Unknown
com.bt.sdk.ServiceException: Unknown exception encountered: java.lang.ExceptionInInitializerError
null
    at com.bt.sdk.capabilities.messaging.oneway.MessagingOneWayManager.<init>(MessagingOneWayManager.java:26)
    at SmsClient$.main(SmsClient.scala:7)
    at SmsClient.main(SmsClient.scala)

Then it means that folks from Lausanne haven’t fixed the bug in Scala Eclipse Plugin yet. It simply doesn’t copy resources on the classpath to the output folder (bin) and the Web21c Java SDK can’t see security.properties and the certificate. For purposes of this exercise you can simply copy content of the resources folder to the bin folder. As hot fix you could also create an ant task similar to the one below. You can make the execution of one or more targets from an Ant build part of the normal build of your project:

<?xml version="1.0" encoding="UTF-8"?>
<project name="copyprops" default="default" >
    <target name="default" >
        <copy todir="${basedir}/bin">
            <fileset dir="${basedir}/src" includes="**/*.*" excludes="**/*.scala"/>
        </copy>
    </target>
</project>

I know that it is very simple demo but I hope it will be of any use for someone. Let me know if you are having any issues with the sample. And remember that Sandbox Usage is restricted.

Popularity: 46% [?]

Share This

There are situations when you can’t put all data which you need in a single instance of a relational database. The reasons may differ. Maybe because it is too much of the data itself. Or there is a problem with network latency of a distributed architecture. Scaling? Or one of many other reasons. The answer is: horizontal partitioning. The process of splitting up your data sets that once has been called partitioning, now, it has a new name: sharding. If your data doesn’t fit on one machine you split it up into segments and each segment is called a shard. It is term used initially at Google but now spreading everywhere.

So… There is a new project Hibernate Shards which is a framework that is designed to encapsulate and minimize complexity of accessing multiple databases by adding support for horizontal partitioning on top of Hibernate Core. It was once the 20 percent project at Google but it is open-sourced now and licensed under the LGPL. If you know the Core Hibernate API you know the Shards API as its implementation hasn’t violated the Core API. Basic assumptions and paradigms for using Hibernate are still valid as Configuration, SessionFactory, and Session objects are almost exactly the same. Interfaces from Hibernate Core:

• org.hibernate.Session
• org.hibernate.SessionFactory
• org.hibernate.Criteria
• org.hibernate.Query

have shard-aware extensions:

• org.hibernate.shards.session.ShardedSession
• org.hibernate.shards.ShardedSessionFactory
• org.hibernate.shards.criteria.ShardedCriteria
• org.hibernate.shards.query.ShardedQuery

The implementations for these four interfaces serve as a sharding engine that knows how to apply an application-specific sharding logic. This logic is a set of rules how data is distributed across its shards. To specify this logic we have to implement the interfaces below:

• org.hibernate.shards.strategy.selection.ShardSelectionStrategy
• org.hibernate.shards.strategy.resolution.ShardResolutionStrategy
• org.hibernate.shards.strategy.access.ShardAccessStrategy

Hibernate Shards comes with a couple of simple implementations of these interfaces. For instance for the Shard Selection Strategy we have choice of sequential (load balanced round robin) or parallel access.

To finish the topic of the sharding logic I shall mention the id generation as well. As the standard database sequences can’t be used in distributed environment we have a choice of two primary key generators:

• ShardedUUIDGenerator - that generates a big random number
• ShardedTableHiLoGenerator - that uses a table in one of the shards to generate the primary key. Obviously it is a single point of failure for our system.

Although project is still in its early stage and even the creators warn about possible glitches this nice wrapper for hiding all the complexity of distributing data around multiple relational databases is very much worth looking at, especially if the Hibernate is already used as the ORM tool of choice.

Popularity: 43% [?]

Share This

Some of you who use Eclipse 3.3 with Sun JVM might notice that it crashes from time to time. Those who have many plugins certainly experienced this more often. Problem and solution for this bug is very trivial but worth mentioning: leaking permgen space. Comes back like November flu! Just to remind the permgen space is a memory for storing “data needed by the virtual machine to describe objects that do not have an equivalence at the Java language level. For example objects describing classes and methods are stored in the permanent generation“. More information about permgen can be found in an article about tuning garbage collection.

To fix the problem edit your eclipse.ini and add following lines:

-XX:PermSize=128M
-XX:MaxPermSize=256M

You might also try Eclipse 3.3.1.1 which includes a fix for the above problem.

Enjoy!

Popularity: 38% [?]

Share This

This blog entry will introduce you how to integrate AStyle into your Ant build.

Short introduction: In my current project I work on software which generates SDKs for multiple languages consuming Web21C services . The SDKs are SOAP webservice clients for multiple languages as Java, Python, PHP or .Net (new are comming soon). XSLT transformations are extensively in use to achieve the goal of generating those SDKs.

The problem is that we are facing a trade off:

• we have ugly formatted XSL to be able to generate correctly formatted output code
• we can have pretty formatted and easy to maintain and read XSL but not correctly formatted output in particular language

Let’s assume that we are not crazy yet and we will not follow the first option. The second option looks better but is still not perfect. We need to plug a tool which will do the dirty job for us: correct indentation, format and beautify our output source. Artistic Style (AStyle) has been selected as the tool as it is small, simple, fast tool and supports both Java and C#.

However the AStyle has couple of downsides:

• doesn’t have easy integration with Ant
• accepts only list of files for transformation, there is no recursive mode

Let’s assume that all source files to be transformed are in folder generated, with multiple subfolders. As there is no integration with Ant we will have to use <exec/> task. Something like:

<property name="astyle.exe" location="${astyle.home}/astyle.exe"></property>
<exec executable="${astyle.exe}" dir="${basedir}">
    <arg value="--mode=java"></arg>
    <arg value="--style=java"></arg>
    <arg value="--suffix=none"></arg>
    <arg value="${sourcefile}"></arg>
</exec>

AStyle accepts following arguments:

astyle [options] SourceFile1.java SourceFile2.java SourceFile3.java [...]

Wouldn’t it be nice to have -R option?

Our first approach could be construction of a space separated list of paths to all files in folder generated and then using this as argument in exec task. We would quickly realize that we have a possible problem with maximum length of a command line executed via exec task which is just 256 characters. Better will be to execute astyle many times, for each file to be formatted. Core Ant task are not enough in this case. We have to use ant-contrib tasks. First copy ant-contrib jar to your ANT_HOME/lib and add in your build:

<taskdef resource="net/sf/antcontrib/antlib.xml" />

It will let you use <for/> task which should help us to solve our problem as it will allow us to iterate over list of paths and execute external command for each one. The final ant task could look like:

<target name="format">
    <for param="file">
        <path>
  	    <fileset dir="${generated}" includes="**/*.java"/>
  	</path>
  	<sequential>
  	    <property name="astyle.exe" location="${astyle.home}/astyle.exe" />
	    <exec executable="${astyle.exe}" dir="${basedir}">
	        <arg value="--mode=java"/>
	        <arg value="--style=java"/>
	        <arg value="--suffix=none"/>
	        <arg value="@{file}"/>
	    </exec>
  	</sequential>
    </for>
</target>

Fist we define a name of the parameter to pass the tokens or files to the sequential, let’s call it just file. Note that for makes use of ant’s macrodef task, so the @{} notation is used for parameter substitution. Next we define path including all java files in generated folder. The last step is to use <sequential/> block where the execution of astyle.exe will happen for each source file defined using @{file} parameter.

Popularity: 40% [?]

Share This

Again about the Mylyn plugin. If you install Eclipse 3.3 Europa there isn’t, due to the Mylar -> Mylyn renaming, the web connector plugin which could be found in the Mylyn 1.0. This plugin is useful when we are working with an unsupported web based bug repository. The plugin still exists but can be found in a separate update site. Simply add new remote update site: http://download.eclipse.org/tools/mylyn/update/extras to find a set of Mylyn plugins including the web connector.

If I find some time I will try to connect to Scrum Works repository using that plugin. If someone has done it already please let me know.

Popularity: 33% [?]

Share This

Our task is to persist a state object called State in a database. State object has globally unique identifier and some other properties. And is serialisable.

Let’s start with creating methods for serialisation/deserialisation:

protected byte[] serialise(T state) {
    try {
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(bos);
        oos.writeObject(state);
        oos.flush();
        return bos.toByteArray();
    } catch (IOException e) {
        throw new IllegalArgumentException(e);
    }
}

protected T deserialise(byte[] byteArray) {
    try {
        ObjectInputStream oip = new ObjectInputStream(new ByteArrayInputStream(byteArray));
        return (T) oip.readObject();
    } catch (IOException e) {
        throw new IllegalArgumentException(e);
    } catch (ClassNotFoundException e) {
        throw new IllegalArgumentException(e);
    }
}

So, we know how to serialise and deserialise object. The next step would be to think where to store that data. I have chosen HSQLDB for that example but it should be very easy to swap to some other db. It should be as easy as changing datasource properties and maybe amend a bit create script. Let’s crate one then for HSQLDB:

create table State (
object_id VARCHAR(32) not null primary key,
object_value LONGVARBINARY
);

Very simple table: object_id which a primary key and objet_value which is a binary representation of the object. And there is beauty of different flavours of databases (not!). We want to store the binary representation in a blob, and for HSQLDB there is no data type as blob. We have to use LONGVARBINARY. We have to be careful when switching to some other vendor and amend the create script. I will skip the part with details how to create schema etc. Contact me if you want more details on that.

Let’s create a datasource for HSQLDB in memory now using org.apache.commons.dbcp.BasicDataSource:

    String driver = "org.hsqldb.jdbcDriver";
    String url = "jdbc:hsqldb:mem:testdb";
    String username = "sa";
    String password = "";

    BasicDataSource ds = new BasicDataSource();
    ds.setDriverClassName(driver);
    ds.setUrl(url);
    ds.setUsername(username);
    ds.setPassword(password);

The datasource will be used by Spring JdbcTemplate (org.springframework.jdbc.core.JdbcTemplate) which we will happily use for interactions with db. You can reuse this object:

JdbcTemplate jdbcTemplate = new JdbcTemplate(ds);

Let’s have a look how to store a serialised object in database (add necessary validation/exception handling):

public void add(T state) {
    Object[] params = new Object[] {state.getId(), new SqlLobValue(serialise(state))};
    int[] types = new int[] { Types.VARCHAR, Types.BLOB };
    jdbcTemplate.update("insert into State (object_id, object_value) values(?, ?)", params, types);
}

Retrieving of that object is slightly more complex. We have to create a callback RowMapper (org.springframework.jdbc.core.RowMapper) first which will map each row from database to our object. This object can be as well reused:

private RowMapper rowMapper = new RowMapper() {
    public Object mapRow(ResultSet rs, int rowNum) throws SQLException {
        byte[] bytes = rs.getBytes("object_value");
        return deserialise(bytes) ;
    }
};

And finally the code for retrieving and deserialising the object from database (validation and exception handling removed):

public T get(String id) {
    Object[] params = new Object[] {id};
    int[] types = new int[] {Types.VARCHAR};
    List results = jdbcTemplate.query("select object_id, object_value from State where object_id=?", params, types, rowMapper);
    if (results.size() == 0)
        return null;
    return (T)results.get(0);
}

Popularity: 42% [?]

Share This

I migrated last week to Eclipse 3.3.0 and during usual routine of installing plugins (must to have: JBossIDE, SpringIDE, Subclipse) I decided to try two plugins I haven’t been using yet. First is a subproject from Eclipse Tools project called Implementors. It fills a gap in standard Eclipse source code navigation which doesn’t really well support working with interfaces. With Implementors plugin it is possible to jump to implementation of an interface and to an interface of implementation. Sounds unimpressive yeah? But it is really useful plugin and I could call it a “Plugin of the Month in Category Simple but Cool”. If I had that category. So, whenever you edit a class which has dependencies defined as interfaces just hit Alt-F3 on a method from an interface to go to implementation. If you have more than one implementation you will have a choice. Ctrl-Alt-F3 will take you to interface. Thanks Andy for introducing that plugin!

The second plugin I wanted to write about is Mylyn formerly known as Mylar. I have heard about this plugin long time ago, talked about it with Tomaz many times but never actually tried it out. My bad and now I regret that. Because I found Mylyn just awesome. In couple of words Mylyn is a task focused ui for Eclipse which filters all unneeded information noise, leaving you only relevant information. It monitors your activity when working on a particular task and creates a context of that task which makes it is very easy to switch between tasks. No more navigating, searching, scrolling again. Mylyn integrates with repositories like JIRA, Bugzilla or Trac. There is also option of using local task repository.

As I use Scrum Works for task management I would like to use that repository with Mylyn but couldn’t find a connector for that one. Anyone solved that problem?

Popularity: 30% [?]

Share This