verture.net — left handed typing since 2001

Magnifying glass for search box

To hide or not to hide 25.08.2002

Had a peek at Adrian Holovatys post about how news sites populate their robots.txt, so I decided to take a peak on some Danish sites, and their robots.txt.

I am not sure if these observations gives me a reason to conclude any mindbending new universal facts, about how Danish news sites filter out web robots from skimming their content. After the big lawsuit against Newsbooster i would have thought that they all had a huge robots.txt-file to keep search engines from indexing their content.

I guess they trust the law then.

› Bio (sort of)

Bio pictureverture.net is the personal website of me, Jonas Voss, and this is my blog. I've lived in Dublin, Ireland from 2005-10, currently live in London, and was born and fully customized in Copenhagen, Denmark. I write about anything that comes to mind. Really.
You can send me an email › if that's how you roll.

Disclaimer: I speak for myself, not my employer. srsly. || This work is licensed under a Creative Commons by-nc-sa License.