Here's a geeky change.

[baldguy]

The robots.txt file on a website tells search engines where they're allowed to search on that site.

Here's DUs robots.txt file:

http://www.democraticunderground.com/robots.txt

# Hello, Robots.

User-agent: *
Disallow: /Templates
Disallow: /workspace
Disallow: /includes
Disallow: /gallery
Disallow: /gallery/pics
Disallow: /gallery/thumbs

Sitemap: http://www.democraticunderground.com/sitemap.php?type=index

Yesterday the whitehouse.gov site had a robots.txt file over 2400 lines long! Here's the link for the list; I won't list it myself.

http://pastebin.com/f18309565

Here's the robots.txt file from whitehouse.gov today, in it's entirety:

http://www.whitehouse.gov/robots.txt

User-agent: *
Disallow: /includes/

Thanks to boing boing
http://www.boingboing.net/2009/01/20/obamas-whitehousegov.html

[ddeclue]

1. Why the hell would it have grown so large?

I understand that this kind of file will grow over time as a website grows and develops but 2400 lines seems ridiculous.

:crazy:

[csziggy]

2. The Bush Webmaster was at least as competent as his other staff &/or appointees n/t

[BadgerKid]

3. It figures.

What better way to make the site not conveniently searchable, while relying on the M$M as a mouthpiece.

But now Obama is returning government back to the people with transparency.

Nice find!