Using delicious in ways you haven't thought of before...

While not technically Java related, I thought I would highlight a neat way to use delicious that you may not have thought of before.

The motivation for this came when my boss came to me and said hey look, man, we need to analyze the data on our forum a little better so that we can better understand what our users are looking for, what problems they are running into etc.

So I'll start there. If you haven't figured it out yet already, I work for Terracotta, and our user forums are over here:

http://forums.terracotta.org/forums/forums/list.page

A blog post for another day will be how we do some neat gymnastics with RSS and Yahoo Pipes to dynamically generate content for our website, but the main thing to note is that the input to that is ... you guessed it delicious.

Now after looking at RSS and delicious, if you think about it for a second, besides being a bookmark manager, actually delicious is really an XML generating engine and database all rolled in one neat little service.

So let me demonstrate how I solved the problem my boss posed to me:

1) Tag items in delicious with special tags. For example, you can see that I have tagged forum posts with tags like "gap-documentation" and "gap-bug" etc. This means for that particular post, the "gap" in our product was either documentation or a bug etc.

Here are my tagged items: http://del.icio.us/tgautier/terracotta%2Bforum

2) Get the RSS feed for all of the posts tagged in this manner. This is easy, I also tag those forum posts with "'terracotta" and "forum" so I just need to get the RSS for the tag intersection of "terracotta+forum":

http://del.icio.us/rss/tgautier/terracotta+forum

3) Generate an XSLT to transform the RSS output into a CSV file. This is a bit tricky, especially if you've never used XSLT before, but once you get the hang of it, not that terribly difficult.

Here's my XSLT:

<?xml version='1.0' encoding='utf-8'?>
<xsl:stylesheet version='1.0' xmlns:xsl='http://www.w3.org/1999/XSL/Transform'
   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
   xmlns:rss="http://purl.org/rss/1.0/"
   xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/"


>
<xsl:output method="text"/>
<xsl:template match="/rdf:RDF">
<xsl:for-each select="rss:item">
<xsl:value-of select="rss:title"/>,<xsl:value-of select="rss:link"/>,<xsl:for-each select="taxo:topics/rdf:Bag/*"><xsl:if test="starts-with(@resource,'http://del.icio.us/tag/al-')"><xsl:value-of select="substring-after(@resource, 'http://del.icio.us/tag/al-')"/></xsl:if></xsl:for-each>,<xsl:for-each select="taxo:topics/rdf:Bag/*"><xsl:if test="starts-with(@resource,'http://del.icio.us/tag/gap-')"><xsl:value-of select="substring-after(@resource, 'http://del.icio.us/tag/gap-')"/></xsl:if></xsl:for-each>,
</xsl:for-each>
</xsl:template>

</xsl:stylesheet>

4) Use the CSV file to generate a pivot table in Excel, and then graph the results.

Here's the result:

Gap                 Total
bug                 8
documentation       12
handhold            6
hib2ndlvlcache      1
jdk16               1
jta                 1
messaging           1
resin               1

So, you can see that while we have been putting a lot of effort into improving our docs, we still have a ways to go! (And of course a lot of people are asking about bugs, which I would expect since that is what Forums are for).

Note: It turns out that the RSS feed is insufficient for my needs. I am now researching the delicious API to get all of the posts. It looks like it will work just fine, but I will need to adjust my XSLT a little since the output is not RSS