Bot Paths

Here’s a cool idea, set up a kind of honeypot for search engines’ bots (and spambots), laid out as a binary tree, and see how the bots behave when crawling the site. The details are written up here. Each page had a node number, written out in English, a comment box (which got populated by spam) and, it seems, a visualization of the populated tree surrounding that node. They’ve tracked how the bots traversed this space over time.

The image here is a thumbnail version of the Yahoo! Slurp bot’s travels over the 2 million pages. Great stuff.

(via information aesthetics)

This entry was posted in Technology and tagged . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  • Tweets

  • Archives