Trawling by

Dale quail.linux at
Wed Jan 5 04:48:48 CST 2005

On Tue, 04 Jan 2005 18:11:52 +1030, Paul Schulz <paul at> wrote:

> Also, can anyone confirm whether MSN is at war with Google?
> MSN seems to be scanning my site, and grabbing the same stuff
> approximately 3 days after the googlebot comes by. They started
> doing this on the 18th of December.

You are lucky you only got MSNbot coming every 3 days, until i blocked
MSNbot from crawling my site via robot.txt they were harrassing my
server daily.  And the stupid thing is a done a search for my web site
in there seach engine and could not find myself.

Damn MSN was just harrassing my server cause it is linux based i guess :(

And googlebot visits my server about once a week and i am indexed in
there serch engine


More information about the linuxsa mailing list