Am I crazy or what?
No not really. You see I just happen to have seen more than one customer run into a cluster deadlock, and it turns out that solving the issue with Terracotta is awesome (actually, Terracotta can automatically detect it in an upcoming version, but shhh don't tell anyone I told you that)
It's funny, really, because I have been hearing this dumb idea that somehow clustered deadlocks with Terracotta is actually this really scary thing -- ooooh watch out for that complicated Terracotta thing, it uses LOCKING (oh gosh) and that can lead to CLUSTERED DEADLOCKS. Oh my. (Anyone know where I can get a clustered deadlock costume, it's almost halloween!)
Really. It's like a bad rumor I keep hearing over and over again. What do they call that when people try to scare other people with rumors that aren't true .... F.....oh nevermind. Here's why deadlocks truly are better with Terracotta:
First, what do we get with Terracotta?
- Kill a JVM, release its lock.
- Kill a JVM, don't lose your state
Why does that matter? Well what do you do when you see a deadlock with a regular Java application? Since it's pretty much hosed, you have to restart it (usually you probably debug the hell out of it first and try to fix the deadlock). But the app is hosed. Unless you happen to have coded a "stateless" app - you've also lost your app state. Bummer :(
Well, not so with an app running on Terracotta. First of all, you don't have to kill the whole app. In fact, if you do actually get a clustered deadlock, you just have to kill one half of the deadlock (because the locks are released, get it?) and the other half will actually get to complete it's operation. How do you do that? Well since the app state is highly available, you can kill any node at will.
So it's simple to resolve a clustered deadlock with Terracotta - just do a rolling restart of every client JVM. That's it. When you hit one half of the deadlock, and kill that JVM, the lock that the other side of the clustered deadlock wants will be freed, and it will go on its merry way.
Now of course, you still need to debug the hell out of your app :). When you fix the app, just update it in place, do another rolling restart, and voila! Fixed deadlock with no downtime.
So to summarize, deadlocks with Java:
- Have to restart
- Lose app state
- Downtime BAD
Deadlocks with Terracotta (e.g. Clustered Deadlocks):
- Rolling restart of application nodes
- Preserve application state
- No downtime GOOD
Tuesday, September 09, 2008
Am I crazy or what?