> Ah, yes. remove your blank lines. A blank line separates sections > based on the user agent. Hm. Still not working. I looked at cnn's robots.txt and I noticed they didn't have multiple directories listed. Just /directory or file.html, not /directory/directory/file.html. Is it ok to put sub folders? Also, I have the spider only looking at www.oshkoshpubliclibrary.org/citydirs. Could that be the problem? -- Jody Cleveland (cleveland@mail.winnefox.org)Received on Wed Jun 25 17:57:35 2003