Scraping Homeland Security

“Scraping” content from web pages means trying to guess at where the content is and making it usable. It’s what the NetFlix plug-in for MT does. After installing the NetFlix plug-in, I decided to adapt it to scrape the Homeland Security Advisory System page over at the Department of Homeland Security.

Unlike the Buffalo weather information over there on the left, which can be updated dynamically by just clicking the “Updated” link, this update is included each time Moveable Type regenerates the index page. It was a fun way to avoid all of the things I need to do this weekend. Click on more, below, if you want to install it on your Moveable Type powered page, and be a part of generating irrational fear.

This is a simple plugin and it is easy to set up.

1. Download the plugin(dhs.pl). FTP it to your site and place it in the directory called “plugins” wherever your Moveable Type is installed. If you find no “plugins” directory in the directory where you find mt.cgi, make it (and chmod both it and the dhs.pl script to 777).

2. Download the images. Easiest is probably to right-click these and use them, or make your own, using the same titles. (Update: like these!)





Place these wherever your index page for MT is.

3. Somewhere in the main template, include something like this:
<img height="40" width="125" border="0" src="<$MTdhsLevel$>">

And you are good to go!

This entry was posted in Uncategorized and tagged . Bookmark the permalink. Trackbacks are closed, but you can post a comment.

10 Comments

  1. Posted 3/19/2003 at 8:56 am | Permalink

    Where are the gifs supposed to be located? I have them in the images dir but get nothing on my page.

  2. Halavais
    Posted 3/19/2003 at 9:14 am | Permalink

    Skip: If you have them in an images dir, you’ll need to specify that in the tag in the template. So, for example, it would look something more like:

    <img height=”40″ width=”125″ border=”0″ src=”images/<$MTdhsLevel$>”>

    (notice the directory “images/” in the src file name)

    Also, remember that you cannot change the names of the images. You can change the images themselves, but must name them the way they are named here (e.g., “dhs-advisory-low.gif”). Let me know if this doesn’t work for you!

  3. Posted 3/23/2003 at 9:52 pm | Permalink

    Thank you mucho for this little doodad!

  4. Posted 3/27/2003 at 11:34 am | Permalink

    i made some MT-sized buttons, based on buttons seen at http://www.maniacalrage.net/ and http://www.antipixel.com/

    you can grab ’em at http://billgrady.com/threat

  5. halavais
    Posted 3/27/2003 at 12:38 pm | Permalink

    Bill: Like your minimalist tags. I think they’d work well for my site, which tends toward the minimal. Folks should check out Mr. Jackson’s images above as well; I find them amusingly well done!

  6. jason b
    Posted 3/29/2003 at 2:51 am | Permalink

    This is a pretty cool little plug-in, given our current state of affairs.

    I went for the minimalist approach with the buttons, but I didn’t like the restriction of using the same extension as the scraped image (I prefer .png). I extended the plugin to allow for an argument “ext”; it defaults to ‘gif’ if not present. If you’re interested, drop me an email.

  7. Posted 3/29/2003 at 11:23 am | Permalink

    Very cool. I had considered doing this, but got lazy :).

  8. Posted 4/2/2003 at 9:46 am | Permalink

    I’ve replaced my version of the script with Jason’s new & improved version. It slices, it dices, it serves up a file with your preferred extension. Everything above remains the same except you can now specify an extension (like png or css) thusly:

    <img src=”/img/buttons/<$MTdhsLevel ext=”png”$>”>

    Cool, n’est pas?

  9. Posted 4/8/2003 at 6:07 pm | Permalink

    Oui, Oui! [Ed: This is spam, duh!]

  10. Posted 4/21/2003 at 2:31 pm | Permalink

    http://www.benhammersley.com/archives/004296.html
    The spammer is still active (17. April 2003), and with the same IP as used here.

5 Trackbacks

  1. By Couchblog on 3/17/2003 at 11:04 am

    Homeland Security Level Plugin – LOL
    Just for fun, em… maybe: Scraping Homeland Security with an MT-Plugin by Alex Halavais (who has a beutiful page indeed).

  2. By this is the samaBlog on 9/7/2003 at 7:32 am

    Hand Rolled
    OK, so I’ve made a few changes to the blogroll, as you no doubt have noticed. First off, I have rearranged everybody’s name by order of length. I know, it’s a stolen concept, Halavais did it first. But I still think it looks cool here. Second, I have c…

  3. By anything but ordinary on 11/18/2003 at 1:12 pm

    Red Alert! Red Alert!
    First came the Homeland Security Plugin, that allows you to display a suitable graphic for the current Homeland Security Advisory on your blog. Now, I present to you… PMS Alert Easily synchronize PMS Alert with her natural monthly cycle. Read…

  4. By Al-Muhajabah's Movable Type Tips on 2/28/2004 at 7:44 pm

    show the Homeland Security threat level
    This is just for fun. Using the dhsLevel plugin, you can display the current threat level from the Department of Homeland Security. What the plugin does is visit the official government site and retrieve the current posted threat level. There’s…

  5. By Minh’s Notes on 3/13/2004 at 9:03 pm

    As it comes to war
    I’m planning to add a host of new features to the next generation of Minh’s Notes. One of them, since the US is going to war, is going to be a little sticker that touts the current HSAS level, courtesy…

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>