How to visualize hot topics from conversations

I've found a new example of a good time-based visualization. Twitterverse shows a tag cloud of the last few hours, based on the world's Twitter conversations. This is one of the things I'll do with email, and it's interesting to see how it works here.

There's a lot of noise in the two-word phrases, with "just realized", "this morning", "this evening" and "this weekend" all showing up as the most common phrases. These don't give much idea of what's on people's minds, but I can imagine you'd need a large stop word system to remove them, and that runs the risk of filtering out interesting phrases too.

A surprising amount of identifiable information came through, especially with single words. For example, xpunkx showed up in the chart, which looked like a user name. Googling it lead me to this twitter account, and then to Andy Lehman's blog. It may just be a glitch of their implementation, but this would be a deal-breaker for most companies if it had been a secret codename gleaned from email messages. Of course, any visualization of common terms from recent internal emails would make a lot of executives nervous if it was widely accessible. Nobody wants to see "layoff" suddenly appear there and cause panic.

It's also surprisingly changeable. Refreshing the one hour view every few minutes causes almost completely different sets of words to appear. Either the phrase frequency is very flat, eg the top phrases are only slightly more popular than the ones just below them, and so they're easily displaced, or their implementation isn't calculating the tag cloud quite in the way I'd expect.

The team at ideacode have done a really good job with Twitterverse, and there's an interesting early sketch of their idea here. Max Kiesler, one of the authors, also has a great overview of time-based visualization on the web with some fascinating examples.

Comments

[Cro-Code]

If interested in word/phrase/wordform frequency analysis, you are welcome to try Textanz text analysis tool.

Various result presentation, friendly interface, fast frequency aclculation algorithm , concordance (context dioctionary), export, charts and more.

See our website for more details. Thank you.

[Pete Warden]

That does look a very interesting tool, thanks for letting me know about it. I'll give it a try, and will let you know how I get on.