Blogumbus Developer Blog

Here’s three unique search concepts to prove that there’s more to search than just Google.

1. Ten by Ten: The Twitter of Mainstream Media

Every hour, the site is updated with the 100 (get it?) most frequently used words and pictures from the last hour’s news.  It then combines these in a kaleidoscope view that you can interact with.  Clicking on a picture gives you a few links to the best articles about the subject.  The result is an “often moving, sometimes shocking, occasionally frivolous, but always fitting snapshot of our world.”  Not only does ten by ten keep a searchable database going back to 2004, but there’s even an API for you to have direct access to this information in your own apps.

Two things about ten by ten really appeal to me.  First, it’s fun to use.  Everyone likes pictures, and everyone likes clicking on stuff.  The animation on the sidebar integrates well into the site, making it fun just to scroll the mouse all over the place.  Second, its got a really clean interface design.  It actually compresses quite a lot of information down into one small space and provides a pretty decent snapshot of the human (or at least American) condition.

I would add one feature, however.  With all this information, why is there no way to compare different time periods?  Even more important than knowing what people find newsworthy right now is knowing how right now compares to the past.  How did reporting about the Iraq and Afghanistan wars change after Obama was elected?  How has it changed now that the Republican primary season is starting? How does the coverage of the Casey Anthony trial differ from the coverage of OJ’s? Without this feature, ten by ten remains little more than a toy.  Albeit a ridiculously awesome toy.

2. Pillbox turns you into a pharmacist without the schooling

Ever found a pile of pills and wondered what they were for?  Is this thing going to fry my brain, cure my cancer, or turn me into a woman?  Fear not because Pillbox will tell you.  You simply describe the pill’s characteristics (size, shape, color, etc.) and Pillbox will tell you what it does.  No medical training required.  Pillbox is provided by the National Library of Medicine and the National Institute of Health.  Unlike most things that come from a government bureaucracy, Pillbox is a real treat.

Pillbox is a great example of the way you can optimize a search interface when you have a very specific and targeted domain.  Pillbox selects five key characteristics of a pill that you can search by (imprint, shape, color, size, and scoring).  To make the search user-friendly, it shows you clear examples of each in picture format.  The results contain detailed pictures of all the possibilities.  A general purpose search engine like Google, or even Google images, could never deliver content to you like this in a format you could trust.  Maybe one day we’ll have millions of awesome little search engines like this linked together by a super-Google.  That, my friends, would be the epitome of search.

3. Kartoo (RIP) the visual search engine

Alas, our third search engine shut its doors in 2010 after ten years of operation.  Kartoo pioneered a unique way of visually representing the user’s search results.  Instead of the standard list, sorted by relevancy, Kartoo arranged search results in a two dimensional grid.  Relevant pages would appear close to the terms that made them come up.

This interface has two benefits over your traditional general purpose search engine layout.  First, there’s a lot of information that can be visually represented by the distance between documents.  It’s very intuitive that two web pages that have similar content should appear close together, or that the most relevant document should appear twice as large as the rest.  Second, search refinement can be done without the use of the keyboard.  As the world moves more and more towards mobile devices with touchscreens, this will become an increasingly important user interface concern.

So if Kartoo was so awesome, why’d they go out of business? There are certainly lots of reasons, but Kartoo had two major design problems.  First, it was slow and kludgy.  They relied on Adobe Flash to render their search results.  With the advent of HTML 5, this is no longer a major concern.  Second, their chosen display format makes it hard to judge when a displayed link is relevant because they have no room to show a preview.  (When you make a search in Google, it will show you a sentence or two from each page that your search terms appeared in.)  This turns out to be incredibly important to the user when they judge the relevancy of the results.  Any future visual search platforms will have to find a way to incorporate this feature.

Google has recently started the Google Wonder Wheel project to experiment with their own version of graphically displayed search results.  (Unfortunately, however, this product is only available in certain regions right now.)

4. A free extra ?!

Don’t forget to check out our unique search design, Blogumbus. Blogumbus is currently in the earliest stages of development and focuses on helping you find blogs with content that appeals to you the most.  We trying to combine the fun of Ten by Ten and the power of Pillbox using the layout scheme of Kartoo.  

So try out the prototype, and remember, we love any and all feedback.

Ever been browsing the web and wanted to find more pages similar to the one you’re on?  Now you can!  In this post, I’m going to show you how to add this functionality to any browser with the Blogumbus bookmarklet.

Don’t click this link: Blogumbus, Instead, right click on it and then click save bookmark. Or, drag it up to your browser’s bookmarks. And that’s all there is to it! Just click the bookmark and it will take you to a Blogumbus page centered on the blog you were just viewing.

As always, let us know what you think!

What is it?

Blogumbus is a free new way to find blogs that’s been specifically optimized for mobile phones. Rather than focusing on search terms (typing complex queries on my Android is so awkward!), we focus on relationships between blogs and allow you to browse those relationships with point-and-click or pinch-and-zoom. Try it out and tell us what you think. We love to hear from our users!

Screenshot of Blogumbus centered on Tech Crunch

When you’re at the home screen, the center blog’s RSS feed displays in the blue box on the right.  Expanding out from the center blog are the five most similar blogs, and expanding off of those blogs are the five most similar blogs to them.  By clicking on a blog, you center it and bring up its RSS feed.  This allows you to surf the blog-o-sphere without ever entering a traditional search query.

So how do we decide if two blogs are similar?  Let’s take Tech Crunch as an example.  Tech Crunch is a blog about the economics of technology companies.  It’s listed in the Technorati top 100 for technology.  We can assume that other blogs listed in the same page are somehow related to Tech Crunch. Unfortunately, Slashdot is not listed because it tends to appeal to a geekier audience. But when other blogs link to Tech Crunch, they also link to Slashdot. Therefore, we can assume that readers of one will be interested in the other and mark them as related.

So go try it out. And remember, we love hearing feedback from our users.

We’re using some relatively obscure technologies that enable us to do things other products simply can’t.

On the backend, we’re using:

1. Neo4j is our database. It is a graph database, which is perfect for the way we structure our search results. (See the wikipedia page for a great description of what a graph database is.) Since the web is a graph, why not actually display and process it that way?Neo4j allows us to do in a single query what would take on the order of 100 queries in a relational database like MySQL. Current page generation times for the server are under 20 ms, and we haven’t even begun to optimize! In the future, we hope to take advantage of graphing algorithms such as the shortest path between webpages to create search results that no one else can.
2. Crawler4j crawls the web for us and finds RSS feeds. So far, we have a “shallow index” that includes about 100,000 blogs. (Essentially, we’re defining a blog as anything with an RSS feed.) By shallow index, we mean that we only index up to 100 pages on any single blog, and then move on to the next blog. There are literally millions of blogs out there, so while we’ve gotten a good start, we still have a ways to go before we’ll be comprehensive. Once we approach that milestone, we’ll go back and do a deep index on the blogs, starting with the most popular.Crawler4j has helped us get up and running extremely quickly, but we are starting to outgrow it and will likely be moving over to Apache’s Nutch fairly soon.
3. Indexing is provided by the Apache Lucene project. This great open source software has all the latest indexing techniques prepackaged. That saves us from having to repeat tons of hard work and allows us to focus on developing a high quality user experience right from the beginning.
4. Finally, our web server is hosted using Jetty. Under most web frameworks, you have to design your application from the ground up to integrate into the framework. In Jetty, it was simply a matter of adding a class to our project files, and suddenly we had an http interface into our database that our webpages could access via AJAX.

On the front end:

1. We’re using the JavaScript InfoVis Toolkit to render the graph structure and allow browsing. The JIT supports some pretty cool features (such as smoothly animating the transition between graphs when you click on a new blog) that we plan on iterating into our next designs. The reason these haven’t been implemented yet is we were having some performance issues on certain browser combinations, and we want to ensure that pages render quickly and smoothly.We also experimented with several other JavaScript libraries including Dracula and Arbor.js, but found that the JIT’s rgraph display structure made the most sense for this application.
2. JQuery and an assortment of other open source plugins including: JQuery Address and JFeed.

It’s truly amazing that we live in a world with so much open source software. It makes rapid prototyping and software development so much more enjoyable to be able to focus on only those aspects that are genuinely novel.

To anyone involved in those projects: Thanks!