Fun with Distributed Programming

Something about the nature of distributed programming makes it quite divisive. You either love it or despise it. It's rare that I've run into someone who is ambivalent about it.

Those that love it, love it because it's hard core. They're proud to know all the ins and outs of dealing with failures, at the system, network, and application level. All of that specialized knowledge is also what turns off the rest of us.

It's kind of like database programming. There are only a select few who really like it. The rest of us only do it because we have to.

Well, I honestly think that Terracotta changes the game. The key is that Terracotta makes distributed programming fun because it takes away most of the distributed programming part, leaving you with just the fun part.

It reminds me of when Linux came out. Everyone loved it because they could just tinker with different schedulers, and not have to think about building an OS from scratch, just to try out a new idea. That's what Terracotta is like. It manages all the hard networking and distributed programming parts, so you get to just play with the algorithms.

Interested? Let's look at a real (if contrived) example. Let's suppose that you have to build the following:

• a service that executes periodically to do some work


• you don't care where this service runs, only that it runs


• it has to run, but one and only one system can run it


• you've got a cluster of n systems, you'd like any one of them to be responsible for running the service

If it were a single JVM, you could do a thousand things, like use a java.util.Timer, or Quartz, or even your own simple Thread with a delay loop in it.

But in a cluster? The choices for synchronizing the behavior of a number of JVMs across a cluster quickly eliminate the fun part, leaving just tedious, boring, and mundane work to be done. Cluster synchronization you're thinking. What should I use? TCP? Multicast? Shared file system locking? A shared database? RMI? JMS? EJBs? Oh dear.

But wait. Terracotta provides synchronization primitives that work across the cluster just like in a single JVM. So that means getting this right in a single JVM means getting it right across the cluster. Could it really be that easy? And fun? Yes!

Let's have a look. For the sake of simplicity, let's do the simple thing. We'll write the delay loop version. We'll implement it as a Singleton that implements Runnable, so we can pass the Singleton to a Thread. Here it is:

public class SimpleWorkRunner implements Runnable
{
    // mark as a Terracotta root
    private static SimpleWorkRunner singleton = new SimpleWorkRunner();

    // singleton pattern - private constructor so there is only one
    private SimpleWorkRunner() { }

    public static SimpleWorkRunner instance() { return singleton; }

    public synchronized void run()
    {
        while (true) {
            // do work
            ...
            try { Thread.sleep(2000); } catch (InterruptedException e) { }
        }
    }
}

That's it! In every JVM, kick off a new thread against the singleton:

...
new Thread(SimpleWorkRunner.instance()).start();

And we're done!

You might have noticed one thing - the run method is synchronized. In a single JVM, this will mean that more than one Thread executed against this Singleton will result in only one Thread winning the synchronization race, and executing the run() method.

In a single JVM, this may not be that important, since there might only ever be one Thread. But with more than one JVM, we will always start at least one Thread per JVM, and that means we have to ensure, per our requirements, that only one Thread ever enters run() at a time.

Terracotta takes care of that for us. We just write the synchronized block, and Terracotta converts that into a cluster lock. And just like in a single JVM, only one Thread - across the cluster - will win the race to enter the run() method.

(Also of note is that this particular implementation assumes that one and only one Thread should assume control and never relinquish it. That was the purpose of the implementation, if you wanted to "bounce" the control around the cluster then we should implement the run method differently depending on the requirements.)

The Terracotta config for this class is trivial. We need to tell Terracotta that the singleton should be a Terracotta root. A Terracotta root will always be the same instance across the entire cluster, which is exactly what we want for a singleton. And we need to autolock the run method so the synchronization is applied to the cluster, not just a local JVM. Here's the config for that:

<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-4.xsd">

  <application>
    <dso>
      <locks>
        <autolock>
           <method-expression>void SimpleWorkRunner.run(..)</method-expression>
        </autolock>
      </locks>
      <roots>
        <root>
          <field-name>SimpleWorkRunner.instance</field-name>
        </root>
      </roots>
    </dso>
  </application>
</tc:tc-config>

We didn't have to worry about the dirty details. Teracotta did. And that means distributed programming becomes fun again!

Find out more:

• Terracotta.org - home page
• Quick Start - download and see the demos
  
• Cookbook - simple recipes that demonstrate Terracotta in action
  

Note that this example is very similar to the Single Resource recipe. Try it out first to get started.

Extra Credit
How does another JVM in the cluster gain control of the task? (Hint: Is it possible for more than one Thread to enter the critical section in run()? In normal Java - no. But what happens in Terracotta with more than one JVM?)