It's the little things that matter...

It's often the little things in a design that make the biggest difference. Sure you have to get the big things right too, but all too often products suffer from a great idea implemented poorly.

So at Terracotta, I often have conversations along these very lines. The job we've carved out for ourselves - clustering the entirety of the Java Virtual Machine, is pretty big. That's why it's such a great place to work - the challenge we face is enormous, and it's enormously fun to tackle it. Let me tell you right now, clustering the VM itself isn't going to happen if you don't get the big ideas right. I'll wager that we have, but only history can prove that one right. But just as important is getting the little things right.

Today I just happened to discover one of those little things. What is it? Well, if you don't already know, Terracotta maintains Object Identity across a cluster of JVMs. That in itself is an amazing feat (no other piece of technology I have ever run into can do this). So Object Identity is the big thing. What's the little thing?

Here goes.

First, my sample code (Main.java):

public class Main
{
    public static final Main instance = new Main();

    private Map<Object, Object> map = new HashMap<Object, Object>();

    public void run() throws Exception
    {
        Object key = new Object();
        Object value = new Object();

        while (true) {
            synchronized (map) {
                map.put(key, value);
            }
            Thread.sleep(500);
        }
    }

    public static void main(String[] args) throws Exception
    {
        instance.run();
    }
}

Those of you not familiar with Terracotta might wonder what's so interesting about this. Well, with Terracotta, you can cluster any Java object, so with the following bit of config, I have done just that:

tc-config.xml:

<tc:tc-config xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.terracotta.org/schema/terracotta-4.xsd">

<application>
  <dso>
    <locks>
      <autolock>
         <method-expression>void Main.run(..) </method-expression>
      </autolock>
    </locks>
    <roots>
      <root>
        <field-name>Main.instance</field-name>
      </root>
    </roots>
  </dso>
</application>
</tc:tc-config>

The map in the Main class listed above is now a clustered map (because the root field, instance, holds a reference to it, and therefore transitively it becomes clustered). Anything I put in the map is clustered (transitively again), meaning every object I put in the map is available to all other JVMs in the cluster. That's pretty cool in it's own right (I happen to think), but how is that different from a normal get/put API in a traditional clustered cache, say EHCache, JCS, or OSCache?

Well, I monitored the number of transactions the little test above generated. How many would you guess? 1? 100? 1 every 500ms?

The answer is actually : 1. Because of Object Identity, after the first iteration through the loop, Terracotta knows that it can optimize out the subsequent calls - there is no need for Terracotta to "re-put" an object for a key that already has that same relationship in the map - so it can save the roundtrip work to the server.

In the clustering world, anything you do on the network is orders of magnitude slower than main memory, so every little thing you can do to keep operations local means a big improvement in latency and throughput. So it may be a minor optimization, but it's got a big effect on the latency and throughput of this application. My application may be trivial, but consider if that map was an HTTP Session Context, or a distributed cache.

Furthermore, this optimization is simply not possible with serialization based solutions (which must implement a copy on read, copy on write strategy), because it's simply not possible for a serialization based approach to track object identity, or changes to objects, and optimize out this kind of a scenario.

However because Terracotta is at the VM level, it knows implicitly when objects change, because of Object Identity, so it is not necessary for a caller of the map to "re-put" objects into the map to make sure it's updated (and it's thus valid to eliminate the subsequent put calls that are superfluous). So in the end - Terracotta would work exactly the same with or without this optimization - the correctness is unaffected by it - but with it, it can, depending on your usage, make your application run orders of magnitude faster.

So, in summary, you gotta get the big things right. Object Identity is the big thing. But it's in getting the little things right - for example optimizing away unnecessary network calls by eliminating redundant map.put() calls, that turn out to take a great idea and make it truly impressive.

Note that I can't take credit for this, since I had nothing to do with creating the feature or even suggesting it. I just happened to have realized that it's trivial to test to see if it's implemented or not, and I did test it and hoped that you would find the results interesting too.

To find out more,

1. Read about Terracotta


2. Check out some bite-sized code samples in the cookbook section


3. Or just download it already :)

Note that the code posted in this demo is 100% runnable - just

1. save it to Main.java and tc-config.xml in a new directory


2. type "javac Main.java",


3. start the Terracotta server - start-tc-server.sh&


4. run the program - dso-java.sh Main