Piotr Woloszyn

First of all it’s Friday afternoon. Yeah! Time for some software musing.

I’ve been profiling recently a legacy J2EE system in an attempt to improve its performance as the system is expected to get much bigger hit in near future. Turned out that only severe architectural changes and essentially rewriting core components would dramatically improve throughput of the system. The underlying issue was the long standing enemy: distributed transactions. This experience led me to thinking more about high performance systems and principles to follow.

Executing a distributed, long transaction and blocking until everything is committed will always be a bottleneck, so we need to make sure to avoid mistakes from the past as we can see everywhere increased business demand for greater scalability. Therefore we can forget about traditional model of the two phase commit. However, in most cases, we can’t trade the reliability and consistency of the system for its performance. So, what are our options? I think the keywords would be:

• loosely coupled
• locally transactional
• operating on local data
• taking advantage of multi-core/machine power

Which leads to:

• data grids
• concurrent programming languages (Erlang, Scala, Clojure, …)
• actor model

To achieve high throughput system that scales but remains consistent and reliable we have to change couple of things. Instead of distributed transactions consider using smaller, independent stages (workflow) that can fail or succeed independently. No need for transaction coordination. Operations by mean of partitioning would be executed where the data regarding the operation exists. The partitioning which is given in data grid solutions like Coherence can be a bit tricky with solutions which don’t provide this feature (Terracotta, at least the version 2.7.3).

Personally for me database is dead.  Personally I would go for Scala.

Popularity: 30% [?]

Share This

I have been asked to help in introduction of a development process for the team working on a greenfield project. The team consist of dozen or so developers with a given task to replace an old legacy system with something which will perform better, will be more flexible and easier to maintain. The existing process that is based on individuals being assigned to never-ending stream of tasks (mostly maintenance, bugfixes and improvements) might work, in my opinion, when system is not in active development stage, therefore not ideal for the greenfield project when there is much bigger density of innovation and collaboration required to succeed.

Interesting task I have to say. I think I will approach this by addressing both project management and engineering requirements by setting up an iterative process based on bits and pieces from Scrum and XP. It shouldn’t be a surprise to anyone involved in the past in Web21C Experiment. The Scrum process will kick off with pretty standard set of features hence I will not bother you with details.

What I want to achieve in field of team organisation is to:

• grow a successful team that delivers business value (doh!) as soon as possible (doh!)
• improve internal and external communication and information flow
• share common understanding of the domain and the system
• learn how to cope with inevitable change
• activate developers to be more encouraged to introduce new ideas and solutions
• put the team in position to be more in the partner relationship with business
• improve motivation and satisfaction from work
• encourage being passionate about technology and sharing that passion

In the same time in field of engineering practices I’ll stand my ground on:

• automation, automation, automation
• continuous integration
• code coverage
• automated integration tests
• patterns
• non mandated on-request pairing (you wonder what it is?)
• lightweight solutions
• shared code-ownership

There is an interesting lean Kanban process on horizon which looks as a good match for the environment however I will not be pushing for this initially. As the process should evolve over time there is always time to navigate original agile scrum process towards something which can be maybe a better choice. Only time will prove.

Comments, advice?

Popularity: 28% [?]

Share This

If you don’t want to use Flink or MacPorts to get you up and running with Erlang environment then you can follow these steps to compile and install Erlang from sources on your Mac.

First make sure you have installed latest Apple’s Xcode Developer Tools which will provide gcc compiler among the other toys.

The next step is to download Erlang sources and man pages. At the time of writing this post the latest version of Erlang was R12B-5. Consult the Erlang download page for details of current release. Let’s get the above mentioned files :

%  wget http://erlang.org/download/otp_src_R12B-5.tar.gz
%  wget http://erlang.org/download/otp_doc_man_R12B-5.tar.gz

Untar the sources and change current directory:

%  tar xzvf otp_src_R12B-5.tar.gz
%  cd otp_src_R12B-5

What we have to do now is to do usual configure/make/make install steps of compilation. To check available configuration options run:

%  ./configure --help

I will configure Erlang with:

%  ./configure --prefix=/sw/erlang --enable-threads
--enable-smp-support --enable-kernel-poll

Next steps are creating the target directory, compilation and installation, uncompressing the man pages:

%  sudo mkdir /sw/erlang
%  make
%  sudo make install
%  sudo cp ../otp_doc_man_R12B-5.tar.gz /sw/erlang/
%  cd /sw/erlang
%  sudo tar xzvf otp_doc_man_R12B-5.tar.gz
%  sudo rm otp_doc_man_R12B-5.tar.gz

The last step will create man folder in /sw/erlang. Everything almost ready. We should add Erlang to the PATH and man pages to the MANPATH i.e. in ~/.bashrc.

export PATH=$PATH:/sw/erlang/bin
export MANPATH=$MANPATH:/sw/erlang/man

The last step is to load the updated .bashrc file:

%  . ~/.bashrc

Everything should be up and running now. Try:

%  man erlang
%  erl

Popularity: 63% [?]

Share This

World awaits Obama’s magic but I’ll write about something slightly more down-to-earth. I’ve been thinking recently about an implementation possibilities for the dynamic cluster configuration. Lets simplify and assume that a system will consist of identical nodes which can take on different responsibilities depending only on a configuration on the node. The communication between nodes would be achieved by using some kind of messaging technology. So, where am I going with this? It would be good to be able to dynamically reconfigure the cluster nodes to serve as required at the time. The cluster can obviously change over time, some nodes might fail, some other nodes might join the cluster and having a cluster manager which can apply configuration to the current cluster shape and size would be great from flexibility and maintainability perspective. And then the Terracotta comes into the picture. Its feature which can be really useful here is set of cluster events which one can listen to:

• com.tc.cluster.event.thisNodeConnected — The local client connecting to a Terracotta server
• com.tc.cluster.event.thisNodeDisconnected — The local client disconnecting from a Terracotta server
• com.tc.cluster.event.nodeConnected — Another client connecting to a Terracotta server
• com.tc.cluster.event.nodeDisconnected — Another client disconnecting from a Terracotta server

To consume the events we will use javax.management classes and first we will get reference to a MBeanServer:

MBeanServer mbeanServer = ManagementFactory.getPlatformMBeanServer();

Then we should get the cluster bean. I created small helper class ClusterUtils as described in the Terracotta JMX Guide:

ObjectName clusterBean = ClusterUtils.getClusterBean(mbeanServer);

We can alrady check what is the identificator of the current node:

String nodeId = mbeanServer.getAttribute(clusterBean, "NodeId").toString();

The only remaining step is registration to the JMX notifications:

mbeanServer.addNotificationListener(clusterBean, new SimpleJmxNotificationListener(), null, null);

Where the SimpleJmxNotificationListener implements javax.management.NotificationListener. We are ready now to consume the events and react accordingly to the changes in the size of the cluster. There is also a JMX Util library that makes using Terracotta Cluster Events even more easy.

By default the cluster nodes will be indentified by strings in format ClientID[SEQ_NUMBER] like ClientID[85]. If we wanted to set different identification in theory we could use -Dtc.node-name=NAME property when starting a given JVM. However there is currently (Terracotta 2.7.1) an open problem with handling of that property which hopefully will be resolved soon.

Everything what is needed to create a dynamic cluster manager is in place now. Cluster manager would be deployed on each node and share own state in Terracotta cluster. It is not difficult to imagine creating a bit of logic to handle connections/disconnections of new nodes in a cluster and dynamically reconfigure the existing nodes to accommodate to the change. There is always a recurring topic of partitioning when using Terracotta, how to follow paradigm of locality of reference when building a low latency, scalable systems. Combination of the cluster management described above with dynamic subscription to messaging topics with sticky routing could be a nice solution to the problem. Just musing…

Popularity: 45% [?]

Share This

Midnight Commander (mc) is one of the tools which I always like to see on *nix systems I use. I just can’t understand why Orthodox file managers like good old Norton Commander on DOS or more up to date excellent Total Commander on Windows are not more popular, but nevertheless I am not here to convince anyone to use them, just want to mention that even if you are a deep command line devotee you could find the tools useful. And certainly if you are deep mouse devotee you should try.

I will explain quickly how to install mc on mac (Mac OSX Leopard 10.5.6) using Fink. Installation with Fink should be painless and this blog post should not exist but I experienced an issue with the process I wanted to share quick solution:

1. If you don’t have Fink yet install it
2. Run:

sudo apt-get install mc

3. Try to start Midnight Commander:

mc

4. Fink should install all dependencies but if you encounter following error go to step 5, if not you are done:

dyld: Library not loaded: /sw/lib/libintl.1.dylib  Referenced from: /sw/bin/mc

Reason: image not found

5. Small dirty hack to fix the previous problem:

sudo ln -s /sw/lib/libintl.3.4.3.dylib /sw/lib/libintl.1.dylib

I know that there are more clean solutions than this one. Share this with me! What’s your favourite OFM?

Popularity: 50% [?]

Share This

I came across a paper from Microsoft on distributed agile development and wanted to share some thoughts from my experience on that topic. It’s not a matter of choice, that’s matter of reality that software teams are more and more often distributed where the scale of distribution may run from:

• homeworkers based in the same city,
• to people based in the same country in different offices/cities
• followed by team members located in the different countries
• then continents and
• time zones.

Luckily we haven’t left this planet yet!

It is nothing unusual to have a team working together day by day dispersed between two, three or four timezones (I’m lucky now with three timezones but with majority of people in the same room). Searching for the new highly skilled professionals, acquisitions in new global markets or outsourcing to cheaper regions could be some of the reasons for increasing distribution.

The physical distance and time difference is not helping to create a funcional agile team. It is actually very far from ideal situation where people sit in the same room, with whiteboard handy and plenty of formal and informal communication going on, following all practices. Broken communication and being unable to apply original agile/xp practices are first problems which come to my mind. It is rather intuitive that dispersed teams will never be as good as team working under the same roof. Or maybe, as I think the last thought goes a bit too far and takes away any hope, the amount of work and skills required to create successful distributed team is bigger for at least an order of magnitude. And again, as always in agile, it is up to each and everyone in the team to make it work.

To fix the broken communication we should actually simply collocate as often as possible with emphasis on planning and release time. It is almost unavoidable. But it is also investment which will pay off when team members will have to work on distance together. People who actually know each other will later on communicate much better. To enable this communication we have to provide right tools: unlimited voice communication with conferencing capabilities, IM clients, tools for remote desktop sharing, access to the same network resources, webcams, hand free headsets. One should also learn how to be open and approachable, that’s a key factor! Pairing is for many people difficult even in the same room and becomes even more difficult on distance, but still possible and beneficial. It’s simply much harder and takes longer to learn how to do it comfortably. Couple of pieces of advice: don’t break something what works by constantly reorganizing teams, don’t loose some people by letting them drift away forgotten. As I already mentioned not all xp/agile practices can be used in original form like pair programming. How you are going to use for instance index cards stuck to a wall with dispersed team? It might not work. But there is always some answer. Maybe a given practice can be slightly modified to fulfill its role, or maybe it can be replaced with some other practice (see paper from Thoughtworks on Subclassing XP) which might work better. To figure out what works and what doesn’t frequent retrospectives with all members is a must. As I said: it is all about communication!

Popularity: 23% [?]

Share This

Me and my new team preparing now to starting a new greenfield project in month or so. Starting something new is very exciting itself and lets us put new processes in place. But how to decide about the processes? Simply by collaborating, discussing and exchanging ideas and previous experience. In this post I will focus on the development/release process and choices we are facing at the moment.

Just to give some perspective… In the past when working on the BT SDK with Web21c team I was using ScrumWorks for managing user stories or, as some other people would call it, features, improvements or bugfixes and mapping those to particular releases. The public releases were very infrequent and scheduled every 3 months. Emergency releases with bug fixes could happen more often. Of course as for an agile team internal releases occurred more often, usually every 2 weeks ended with an acceptance session with a product owner. The Subversion has been used as the source repository, with all active development happening usually in parallel in the trunk. Some time before release the code was frozen in a branch by copying an active trunk and only bug fixes were applied to the release branch, subsequently merged back to the trunk. I could call this process “Develop in trunk, release from a branch”.

At my current project we use only JIRA for managing bugs/features/improvements and matching it to particular releases and the releases will occur much more often (every week or two) with possible multiple emergency releases in meantime. Source is kept in Subversion as well. The team is not calling itself “agile” however I think it has to be very agile to cope with the schedule and a right process should be established to support it. It could look like:

Development time

1. In JIRA we create a bunch of tickets for new features, bug fixes or improvements. A developer is assigned (hey, who likes that phrase?) a task to work on.
2. Before starting work a new branch is created from trunk with the JIRA number as a name
3. Work on that task commences and all changes are committed to the new branch. If there are several dependant tickets then JIRA should have a parent ticket created and all dependant tickets assigned as children. It resembles higher level user story with tasks from Scrum.

Build/Release time

1. In JIRA a release or version will be set up and given a name (i.e. 1.0.1). This will contain a list of tickets that are to be used for this release
2. A new branch will be created in SVN from trunk, matching the name of the JIRA release.
3. The branches that match the JIRA tickets for this release are merged into the new release branch.
4. A continuous integration build is pointed to the newly created release branch.
5. The build should pass unit tests, integration tests, automated acceptance tests, is then deployed to UAT environment for final user acceptance tests.
6. When ready for release, a tag is created from the branch matching the JIRA release name. The software is deployed to production.
7. The code from the tag is now merged back to trunk

It is clearly “Develop in branches, release from trunk”. Sounds weird at the beginning. Have to admit that I didn’t use exactly this process before but my first impression is that it is really structured methodology which:

• gives superb view what actually is currently deployed (via mapping tickets to releases in JIRA)
• allows adding/removing changes easily – we can always take a previous tagged version and apply selected modification by merging
• supports external auditing

In the same time my gut feel tells me that:

• merging before release might be a nightmare (volunteers for a release manager?)
• the process seems to be too regulated, “boxed”, “walled”

It will very much depend on the release schedule and how often the changes will be released which means how often the trunk will be up to date with the latest code. Let’s assume we have biweekly releases. It will mean that at the average a developer will be working with a one week old code. With an application in a maintenance mode with occasional bug fixes which doesn’t change dramatically it would be acceptable but taking as example a new project with many contributors, dependencies between components and a lot of expected changes in design and implementation I think that might be problematic.

It really doesn’t encourage refactoring neither. Just imaging what would happen if two branches taken from the same version of TRUNK were exposed to wider and more intensive refactoring? And for me refactoring is a necessary factor which means continuous and gradual code quality improvement leading to system quality improvement. I know, we should do everything right first time. Why I didn’t think about this before?

Other downside which I can see is that we might completely lose transparency and visibility what is happening with the code. People will commit to “own” branches without a continuous integration. I have to say that I like to monitor changes to current source code via email from continuous integration with last check-in comments. It gives me a way of quickly assessing if the ongoing development is related to my current task, I can merge incoming changes one by one with my changes which means that I don’t do something which someone already has done or something which doesn’t make sense any more.

So, I have mixed feelings. The biggest advantage of the process I can see is easy adding/withdrawing pieces of functionality at short notice. And it (at least in theory, not mentioning the merging hell) fulfils the role very well. The process actually resembles the way how open source communities work. Each contributor branches from trunk and does development on his/her own without breaking anyone else code. When feature is ready then it is (after code review) merged back to trunk or release branch.

It appears that the problem here is caused by subversion not being the best at the merging job. So as we can see a good tool for merging is essential here to make the life easier. It looks like open source guys moved towards GIT and BAZAAR now with GIT getting a lot of traction as it has support for SVN as well.

The process described is not final and would look into refining it to better support given development/release practises. If you have suggestions, comments or you have been using similar process please free to contribute and share.

Popularity: 42% [?]

Share This

I came across a strange behaviour of Selenium and wanted to share my pain with someone really. But first couple of words of introduction. I use Selenium in my current webapp project which I am working on. The application uses quite a lot of javascript (Dojo framework mostly) which can be tested using D.O.H. framework but the Selenium is giving us the final sanity checks, acts as integration test framework and may be used by product owner in an agile environment to drive acceptance on user stories via automated acceptance tests.

Selenium is an excellent package. I really love it. Just to mention integration with various browsers, Selenium IDE, support for integration with build (Selenese ant task) and really easy way of creating and storing the tests as HTML. It is a framework which cannot be ignored by anyone working on webinterfaces.

There is previously mentioned selenese ant task which helps with integration of the Selenium tests and an ant build. You can grab the jar from the Selenium RC distribution.

However as most of the things around us it is not a perfect software. Anyone had the famous “Permission denied to get property Location.href”? Cross domain scripting issues?
In my case I figure out two reasons for above problem and I wanted to share it:

Problem 1:

Let’s look at the following selenium command:

• open
• http://host/some/url

Looks good, doesn’t it? So it might work, but it can as well break on another box with the “Permission denied to get property” and some other scary warnings. The solution in this case was simply to remove the host definition from url leaving just:

• open
• /some/url

Problem 2:

One of the pages I was testing had redirection in case of the authorization violation. So esentially after single HTTP request the browser was receiving single response with redirection and then performed another request to the URL defined in the redirection. Selenium couldn’t handle that correctly. But again, it wasn’t deterministic, it could work on one box but fail on another. Timing issues etc. Hate that. Neither of the combinations worked in consistent way: openAndWait, open + pause.

The solution in this case required change in the webapp to remove usage of the redirection (which I believe was an ugly way of dealing with the problem anyway).

Hope it will help someone with similar issues. If you know another tricks or something to avoid when using Selenium please contact me or leave a comment here.

Popularity: 94% [?]

Share This

It is an example how to use the Web21c SDK from a Scala application.

First couple of words about Scala. Scala was created in 2001 at the École Polytechnique Fédérale de Lausanne (EPFL) in Switzerland. It is platform independent, multi-paradigm language which integrates features of object-oriented and functional programming, is type-safe (but you don’t have to define redundant types of information) and extensible (good candidate for DSLs). It brings a lot of very nice syntactical sugar as well! As Scala smoothly integrates with Java and .Net worlds we can run Scala applications on JVM or .NET CLR and easily reuse existing libraries for those platforms.

One of those libraries is Web21c SDK which gives us easy way of integrating communication services and comes in couple of flavours: SDK for Java, .Net, Python and PHP. We could integrate Scala with Java or .NET SDK but for the purpose of this example we will see how easy it is to incorporate Java SDK into our Scala application.

Let’s assume we use Scala plugin for Eclipse and we created a new Scala project.

We can get the latest Web21c SDK for Java and before proceeding to next steps we have to register our application and obtain own certificate for sandbox server which is free. Then we will create two folders: lib and resources in our project and we will unzip content of the Web21C-JavaSDK-5.0.zip and copy content of the lib folder from SDK to our lib folder. Let’s copy the Web21C-JavaSDK-5.0.jar there as well. We will copy previously obtained certificate to the resources folder and we will create a properties file called security.properties where you can store name of your cert and its password:

org.apache.ws.security.crypto.provider=com.bt.security.PKCS12Crypto
org.apache.ws.security.crypto.merlin.keystore.type=pkcs12
org.apache.ws.security.crypto.merlin.keystore.password=YOUR_CERTIFICATE_PASSWORD
org.apache.ws.security.crypto.merlin.alias.password=YOUR_CERTIFICATE_PASSWORD
org.apache.ws.security.crypto.merlin.file=YOUR_CERTIFICATE.pfx

constants.provider=com.bt.sdk.common.SandboxConstants

Next from Eclipse in the project Properties/Java Build Path/Source/Add Folder… we will add resources folder to the classpath and all jars from lib folder to the Properties/Java Build Path/Libraries/Add JARs… to place them on the classpath as well. We should be good to go.

Then we will create the Scala singleton where we will use directly Java objects. For example Manager classes for one of the Web21c services, for instance One Way Messaging to create a simple sms client (src/SmsClient.scala):

import com.bt.sdk.capabilities.messaging.oneway.MessagingOneWayManager

object SmsClient {
    def main(args: Array[String]) {
        new MessagingOneWayManager().sendMessage("tel:44xxxxxxxxx", "Hello World")
    }
}

Or even simpler:

import com.bt.sdk.capabilities.messaging.oneway.MessagingOneWayManager

object SmsClient extends Application {
    new MessagingOneWayManager().sendMessage("tel:44xxxxxxxxx", "Hello World")
}

Is that close to the famous “one line of code”?

Our project’s structure should look more or less like:

+ Web21cAndScala
    L---+ src
        L--- Web21c.scala
    L---+ bin
        L--- //compiled classes
    L---+ lib
        L--- Web21C-JavaSDK-5.0.jar
        L--- //other jars from lib folder
    L---+ resources
        L--- sandbox.pfx
        L--- security.properties

We are ready to try if it works. Try Run As… Scala Application. Great if it works! If you get following error:

Exception in thread "main" Reporting Service: Unknown
com.bt.sdk.ServiceException: Unknown exception encountered: java.lang.ExceptionInInitializerError
null
    at com.bt.sdk.capabilities.messaging.oneway.MessagingOneWayManager.<init>(MessagingOneWayManager.java:26)
    at SmsClient$.main(SmsClient.scala:7)
    at SmsClient.main(SmsClient.scala)

Then it means that folks from Lausanne haven’t fixed the bug in Scala Eclipse Plugin yet. It simply doesn’t copy resources on the classpath to the output folder (bin) and the Web21c Java SDK can’t see security.properties and the certificate. For purposes of this exercise you can simply copy content of the resources folder to the bin folder. As hot fix you could also create an ant task similar to the one below. You can make the execution of one or more targets from an Ant build part of the normal build of your project:

<?xml version="1.0" encoding="UTF-8"?>
<project name="copyprops" default="default" >
    <target name="default" >
        <copy todir="${basedir}/bin">
            <fileset dir="${basedir}/src" includes="**/*.*" excludes="**/*.scala"/>
        </copy>
    </target>
</project>

I know that it is very simple demo but I hope it will be of any use for someone. Let me know if you are having any issues with the sample. And remember that Sandbox Usage is restricted.

Popularity: 46% [?]

Share This

If you have to use Outlook I believe you deserve better search features than the features available for you by default. Many of you know the great search plugin Lookout. However after migrating to Outlook 2007 I was unable to successfuly install and use it. Instead after restarting Outlook I got error message:

Sorry! It looks like another Outlook Plugin has installed an unofficial version of the Outlook libraries which breaks Lookout. Lookout will not be able to load. […].

But it looks like there is a way to install Lookout on Outlook 2007. After closing Outlook run from command line:

cd  %SYSTEMROOT%\assembly\GAC

rename Microsoft.Office.Interop.Outlook Microsoft.Office.Interop.Outlook.OLD

type nul > Microsoft.Office.Interop.Outlook

rename Policy.11.0.Microsoft.Office.Interop.Outlook Policy.11.0.Microsoft.Office.Interop.Outlook.OLD

After restarting Outlook everything should work. Don’t blame me if it doesn’t. But in case it doesn’t you can revert changes executing following commands:

cd  %SYSTEMROOT%\assembly\GAC

rename Policy.11.0.Microsoft.Office.Interop.Outlook.OLD Policy.11.0.Microsoft.Office.Interop.Outlook

del Microsoft.Office.Interop.Outlook

rename Microsoft.Office.Interop.Outlook.OLD Microsoft.Office.Interop.Outlook

Hope it helps!

Popularity: 100% [?]

Share This