High transactional throughput and global transactions
First of all it’s Friday afternoon. Yeah! Time for some software musing.
I’ve been profiling recently a legacy J2EE system in an attempt to improve its performance as the system is expected to get much bigger hit in near future. Turned out that only severe architectural changes and essentially rewriting core components would dramatically improve throughput of the system. The underlying issue was the long standing enemy: distributed transactions. This experience led me to thinking more about high performance systems and principles to follow.
Executing a distributed, long transaction and blocking until everything is committed will always be a bottleneck, so we need to make sure to avoid mistakes from the past as we can see everywhere increased business demand for greater scalability. Therefore we can forget about traditional model of the two phase commit. However, in most cases, we can’t trade the reliability and consistency of the system for its performance. So, what are our options? I think the keywords would be:
- loosely coupled
- locally transactional
- operating on local data
- taking advantage of multi-core/machine power
Which leads to:
- data grids
- concurrent programming languages (Erlang, Scala, Clojure, …)
- actor model
To achieve high throughput system that scales but remains consistent and reliable we have to change couple of things. Instead of distributed transactions consider using smaller, independent stages (workflow) that can fail or succeed independently. No need for transaction coordination. Operations by mean of partitioning would be executed where the data regarding the operation exists. The partitioning which is given in data grid solutions like Coherence can be a bit tricky with solutions which don’t provide this feature (Terracotta, at least the version 2.7.3).
Personally for me database is dead. Personally I would go for Scala.
Popularity: 30% [?]
