Thursday, October 04, 2007

Reimplementing RMI

Today, on our forums, a user asked how to implement an RMI like solution:


My clustered app will be a client of an external service (via a socket connection) where only ONE node in the cluster will maintain the connection. All the other nodes will need to utilize the external service via the node connected to it.


An interesting puzzle. It's basically RMI. Should one reimplement RMI with Terracotta? That wasn't the point -- I can't resist a challenge like this, so I whipped up a solution. Ultimately, it ended up being a combination of a queue to send the method invocation, and some wait/notify to wait for the response.

As I was chatting about the solution with DSO Guy Steve Harris I remarked that I often come across questions of the sort "how do I do this in Terracotta?" I said that the answer was remarkably easy - though I think it just doesn't occur to people how easy it really is.

The great thing about Terracotta is that it really gives you just Java. By extending the Java Memory Model, Terracotta makes sure that memory and synchronization work exactly the same as in a single JVM. So the answer to the question "how do I do this in Terracotta?" is extremely easy: How do you do this in a single JVM with multiple threads? Whatever your answer is, that's the answer to doing it with Terracotta.

To illustrate this point, let's see how I solved the RMI question. Basically, I implemented a Dynamic Proxy (with some help from Hung on writing a Dynamic Proxy here which neatly wraps up the guts of the implementation so it can be wrapped around any implementation.

As a result of using Dynamic Proxy, we already get a Method and Arguments that we have to handle, so to execute this method on a specific node is the same as executing it on a separate thread. To do that, use a queue. And to signal the response, use wait and notify.

Here's the code:
import java.util.concurrent.*;
import java.lang.reflect.*;

public class RemoteInvoker implements InvocationHandler, Runnable
{
private BlockingQueue<MethodArguments> queue = new LinkedBlockingQueue<MethodArguments>();

private final Object instance;

public RemoteInvoker(Object instance)
{
this.instance = instance;
start();
}

private static class MethodArguments
{
public final Object proxy;
public final Method method;
public final Object[] args;
private MethodResult result;

public MethodArguments(Object proxy, Method method, Object[] args)
{
this.proxy = proxy;
this.method = method;
this.args = args;
}

public synchronized MethodResult getResult() throws InterruptedException
{
while (result == null) { wait(); }
return result;
}

public synchronized void setResult(MethodResult result)
{
this.result = result;
notify();
}
}

private static class MethodResult
{
public final Object object;
public final Exception exception;

public MethodResult(Object object, Exception exception)
{
this.object = object;
this.exception = exception;
}
}

private void start()
{
Thread t = new Thread(this);
// t.setDaemon(true);
t.start();
t.start();
}

public void run()
{
synchronized (instance) {
System.out.println("I am servicing requests...");

MethodArguments arguments;
while (true) {
try {
arguments = queue.take();
try {
Object value = arguments.method.invoke(instance, arguments.args);
arguments.setResult(new MethodResult(value, null));
} catch (Exception e) {
arguments.setResult(new MethodResult(null, e));
}
} catch (InterruptedException e) {
// do nothing
}
}
}
}

public Object invoke(Object proxy, Method method, Object[] args) throws Throwable
{
MethodArguments arguments = new MethodArguments(proxy, method, args);
queue.put(arguments);
MethodResult result;

result = arguments.getResult();
if (result.exception != null) {
throw result.exception;
}

return result.object;
}

And here's how to use it:
import java.lang.reflect.*;

public class Test implements TestInterface
{
int counter = 0;

public int count(int count)
{
System.out.println("Incrementing counter by: " + count);

counter += count;
return counter;
}

public static void main(String[] args)
{
Test t = new Test();
RemoteInvoker invoker = new RemoteInvoker(t);
TestInterface proxy = (TestInterface) Proxy.newProxyInstance(t.getClass().getClassLoader(), new Class[] { TestInterface.class } , invoker);
System.out.println("Proxy returns: " + proxy.count(3));
}
}

What I really like about this solution, and Terracotta in general, is that it works exactly the same in a single JVM as many JVMs. The key difference to many JVMs is that by clustering the Queue, the execution is transferred across physical JVMs, and the wait/notify sends the response back - no different than cross-thread communication in a single JVM.

For the complete instructions on running this with Terracotta, read the whole thread.

Btw, if you read the code, maybe you were wondering why run() method is synchronized against instance. Well, getting back to the request from the user, the original requirement was that only one instance can be "connected" - in other words servicing requests. The synchronized makes sure only one instance is servicing the queue - whether it's a single JVM or many JVMs.

9 comments:

Unknown said...

Cool post.

It is a bit similar to the stuff that I wrote about some time ago: http://jonasboner.com/2007/07/27/the-power-of-sockets-and-dynamic-proxies/

But much much more elegant. Terracotta is a bit like cheating though :-) Makes stuff almost too easy.

The hard days of "do-it-all-yourself" are over, and strange as it sounds, I am not missing them. Getting old I guess. :-)

/Jonas

Taylor said...

Jonas,

It seems my lot in life is to follow in your footsteps ;-)

Philip Wu said...

Forgive me if I am off track. It's been awhile since I've done any multi threaded programming...

synchronizing on instance object means, that only one node could be connected at any given time. But once the lock is relinquished, wouldn't all the other nodes execute it as well? So its not really RMI (where I would expect only one node to be executed), but its still broadcasting and instead of executing in parallel, they're executing in sequence.
Is that right?

Philip Wu said...

nevermind it kinda is like RMI. It's not really broadcasting because once the item is taken off the queue, no other node can process it. Similar to JMS

Unknown said...

Very interesting post. However, when I tried to implements this, the throwing of exceptions seems problematic in that TC asks for the exception being thrown to be included in the additional-boot-jar section of tc-config.xml. I added InvocationTargetException to the additional boot jar, but realize this isn't enough, and adding all the exceptions that could be thrown is infeasible.

Any thoughts on this.

I get the following error for example:

*******************************************************************************
Attempt to share an instance of a non-portable class referenced by a portable cl
ass. This
unshareable class must be in the DSO boot jar. Please add this class to the boot
jar
configuration and re-create the DSO boot jar.

For more information on this issue, please visit our Troubleshooting Guide at:
http://terracotta.org/kit/troubleshooting

Referring class : java.lang.reflect.InvocationTargetException
Referring field : java.lang.reflect.InvocationTargetException.target
Thread : Thread-20
JVM ID : VM(2)
Class to add to boot jar: java.lang.NullPointerException

Action to take:

1) Reconfigure and rebuild the boot jar
* edit your tc-config.xml file
* locate the [dso] element
* add this snippet inside the [dso] element

[additional-boot-jar-classes]
[include]java.lang.NullPointerException[/include]
[/additional-boot-jar-classes]

* if there is already an [additional-boot-jar-classes] element present, simpl
y add
the new includes inside it
* Recreate the boot jar by running the 'make-boot-jar' tool in the bin/ direc
tory

It is possible that this class is truly non-portable, the solution is then to
mark the referring field as transient.


*******************************************************************************

Anonymous said...

威創牙醫診所除了提供優質的植牙技術外還提供假牙|矯正|牙周病治療,是值得您信賴的牙醫診所

獅王紋身工作室提供專業的無痛刺青技術,獅王紋身在世界TATTOO大賽中,獲獎無數,獅王紋身給您最時尚的作品。

陳駿逸皮膚科診所提供了治療痘痘的服務,皮膚雷射權威,包括雷射脈衝光除斑等,讓您回復青春蘋果臉。

ck皮件處理棧提供專業洗包包|洗鞋子|各式皮件修理保養疑難雜症都有服務,清洗包包專門店讓您的包包、鞋子、永遠保持最新的況態唷。

杏儒中醫診所提供了糖尿病的治療。

seo大師e王國幫您的網站輕鬆在您的行業裡站上第一頁,e王國的關鍵字行銷是您的好幫手,包括關鍵字自然排序、都能讓您獲得完美的效果,以目前的網路行銷不外乎是關鍵字自然排序為主、而關鍵字行銷seo又是e王國的強項讓e王國幫您征服網海。

Anonymous said...

head junior tennis racketwilson tennis racquet
wilson tennis racket
head tennis racketbabolat tennis racket
Lacoste polo shirtschaussure nike Ugg Bootspuma CATblack ugg boots
nike max ltd
ed hardy womens ed hardy sunglasses ed hardy mens

Anonymous said...

ghb I like your blog. Thank you. They are really great . Ermunterung ++ .
Some new style Puma Speed is in fashion this year.
chaussure puma is Puma shoes in french . Many Franzose like seach “chaussure sport” by the internet when they need buy the Puma Shoes Or nike max shoes. The information age is really convenient .
By the way ,the nike max ltd is really good NIKE air shoes ,don’t forget buy the puma mens shoes and nike air max ltd by the internet when you need them . Do you know Nike Air Shoes is a best Air Shoes . another kinds of Nike shoes is better . For example , Nike Air Rift is good and Cheap Nike Shoes .the nike shox shoes is fitting to running.
Spring is coming, Do you think this season is not for Ugg Boots? maybe yes .but this season is best time that can buy the cheap ugg boots. Many sellers are selling discounted. Do not miss . Please view my fc2 blog and hair straighteners blog.
.thank you .

I like orange converse shoes ,I like to buy the cheap converse shoes by the internet shop . the puma shoes and the adidas shoes (or addidas shoes) are more on internet shop .i can buy the cheap nike shoes and cheap puma shoes online. It’s really convenient.
Many persons more like Puma basket shoes than nike air rift shoes . the Puma Cat shoes is a kind of Cheap Puma Shoes .
If you want to buy the Cheap Nike Air shoes ,you can buy them online. They are same as the Nike Air shoes authorized shop. Very high-caliber Air shoes and puma cat shoes . the cheap puma shoes as same as other.

Anonymous said...

池袋 風俗
渋谷 風俗
新宿 風俗
アダルトDVD
av 写真
アダルトDVD
アンドロペニス
オナホール
コンドーム
ローション
SM 通販
セクシー下着
男性下着
メンズTバック
大規模修繕
決済代行
SEO
SEO
Hicht hetta hoop
カード決済
FX 比較ランキング
クレジットカード 比較
仔犬
子ウサギ
仔ウサギ
アダルトショップ
アダルトグッツ
ゴールドカード ランキング
オナホール