So you want buzz, eh? Do it in style with del.icio.us, pipes, and widgets

It seems buzz is all the buzz these days.

If you don't want to be left behind in this day and age of blogs, mashups, rss, widgets, ajax and yes, even comets, you've come to the right place to generate some buzz, and you won't even need a full-time intern to do it!

Today I am going to show you how to put together a complete buzz generating engine, in just three simple steps, that is fully distributed and collaborative, but more importantly is completely web 2.0 and buzzword compliant. Sound good? Good. :)

So here we go. Before we start, let's ground this exercise in something concrete. Imagine a marketing droid has come to you with the preceding two paragraphs, and you need to understand what in the heck he actually means.

If you page on over to www.terracotta.org, scroll on down past all the stuff about Terracotta, and have a look at the section marked "Terracotta Buzz". Ok, I understand you're a bit lazy with the ol' mouse click, so I've included a screenshot to the right for your convenience. It says "Terracotta Buzz" and then a list of links. What the marketing droid wants is:

• A dynamic list of links, kept up to date
• Relevant content, not just random links (try searching for Terracotta, you'll see the term is a bit generic so it can be difficult to discern real buzz from somebody's trip to China)

What you want, because you don't have any spare time in your day, is a process that:

• Is easy to maintain, requiring no intervention on your part
• Is up-to-date (what good is buzz if it's not up to date?)
• Is distributed and collaborative (you won't be the only person contributing, that way you can get on with your life)

Got it?  Ok here we go.   Three easy steps.  I wouldn't lie to you, now would I?

Step 1 - Add the Widget to your page

On your buzz page, setup an RSS widget that reads an RSS feed. When users browse your site, the RSS widget reads the RSS feed and gives them the most up to date info.  Did you notice the Terracotta Buzz on the right of my blog?  That's an RSS widget BlogSpot gives you, putting it on your blog page is trivial.  

Step 2 - create a Yahoo pipe

Set up a Yahoo account and create a Yahoo pipe that aggregates RSS feeds from delicious users that are going to generate the buzz for you. Here's the Terracotta Buzz Pipe.

It's the heart of this process, so I'll explain what it does:

• Aggregate content from various sources, including delicious accounts of contributors to Terracotta, RSS feeds from people that blog about Terracotta (like myself)
• Remove duplicate entries
• Publish one aggregate feed as the result

Step 3 - Tag links using del.icio.us

After creating the Yahoo Pipe to aggregate del.icio.us links, now all you have to do is populate those links.  As the tagger, you want to create a unique set of tags that can be used in the Yahoo!  Pipe feed to pull in just the Buzz links.  I use the special tag "forterracotta", which you can see by browsing my links here:

• tgautier's forterracotta del.icio.us tag feed

That is all there is to it.  There's an optional step to burn your feed using Feedburner which adds an extra level of abstraction above the Yahoo! Pipes feed, and gives you the ability to track subsribers.  It's a nice touch, but not strictly necessary.

Summary

Putting it all together, here is the complete diagram of the process:

Using delicious in ways you haven't thought of before...

While not technically Java related, I thought I would highlight a neat way to use delicious that you may not have thought of before.

The motivation for this came when my boss came to me and said hey look, man, we need to analyze the data on our forum a little better so that we can better understand what our users are looking for, what problems they are running into etc.

So I'll start there. If you haven't figured it out yet already, I work for Terracotta, and our user forums are over here:

http://forums.terracotta.org/forums/forums/list.page

A blog post for another day will be how we do some neat gymnastics with RSS and Yahoo Pipes to dynamically generate content for our website, but the main thing to note is that the input to that is ... you guessed it delicious.

Now after looking at RSS and delicious, if you think about it for a second, besides being a bookmark manager, actually delicious is really an XML generating engine and database all rolled in one neat little service.

So let me demonstrate how I solved the problem my boss posed to me:

1) Tag items in delicious with special tags. For example, you can see that I have tagged forum posts with tags like "gap-documentation" and "gap-bug" etc. This means for that particular post, the "gap" in our product was either documentation or a bug etc.

Here are my tagged items: http://del.icio.us/tgautier/terracotta%2Bforum

2) Get the RSS feed for all of the posts tagged in this manner. This is easy, I also tag those forum posts with "'terracotta" and "forum" so I just need to get the RSS for the tag intersection of "terracotta+forum":

http://del.icio.us/rss/tgautier/terracotta+forum

3) Generate an XSLT to transform the RSS output into a CSV file. This is a bit tricky, especially if you've never used XSLT before, but once you get the hang of it, not that terribly difficult.

Here's my XSLT:

<?xml version='1.0' encoding='utf-8'?>
<xsl:stylesheet version='1.0' xmlns:xsl='http://www.w3.org/1999/XSL/Transform'
   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
   xmlns:rss="http://purl.org/rss/1.0/"
   xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/"


>
<xsl:output method="text"/>
<xsl:template match="/rdf:RDF">
<xsl:for-each select="rss:item">
<xsl:value-of select="rss:title"/>,<xsl:value-of select="rss:link"/>,<xsl:for-each select="taxo:topics/rdf:Bag/*"><xsl:if test="starts-with(@resource,'http://del.icio.us/tag/al-')"><xsl:value-of select="substring-after(@resource, 'http://del.icio.us/tag/al-')"/></xsl:if></xsl:for-each>,<xsl:for-each select="taxo:topics/rdf:Bag/*"><xsl:if test="starts-with(@resource,'http://del.icio.us/tag/gap-')"><xsl:value-of select="substring-after(@resource, 'http://del.icio.us/tag/gap-')"/></xsl:if></xsl:for-each>,
</xsl:for-each>
</xsl:template>

</xsl:stylesheet>

4) Use the CSV file to generate a pivot table in Excel, and then graph the results.

Here's the result:

Gap                 Total
bug                 8
documentation       12
handhold            6
hib2ndlvlcache      1
jdk16               1
jta                 1
messaging           1
resin               1

So, you can see that while we have been putting a lot of effort into improving our docs, we still have a ways to go! (And of course a lot of people are asking about bugs, which I would expect since that is what Forums are for).

Note: It turns out that the RSS feed is insufficient for my needs. I am now researching the delicious API to get all of the posts. It looks like it will work just fine, but I will need to adjust my XSLT a little since the output is not RSS