Skip to main content.
home | support | download

Back to List Archive

Re: Does the <!-- Swishcommand noindex --> work whe

From: <Cleveland(at)not-real.mail.winnefox.org>
Date: Fri Jun 27 2003 - 13:11:51 GMT
Hello,

> Try this out (changing "site" to be your site).  This is 
> from the WWW::RobotRules man page. 
> 
> moseley@bumby:~/apache$ cat r.pl

Ok, I made that file, ran it, and got this output:
[cleveland@storm cleveland]$ perl r.pl
==========
User-agent: *
Disallow: /citydirs/1857/1857full.pdf
Disallow: /citydirs/1866/1866full.pdf
Disallow: /citydirs/1868/1868full.pdf
Disallow: /citydirs/1869/1869full.pdf
Disallow: /citydirs/1872/1872full.pdf
Disallow: /citydirs/1876/1876full.pdf
Disallow: /citydirs/1879/1879full.pdf
Disallow: /citydirs/1880/1880full.pdf
Disallow: /citydirs/1883/1883full.pdf
Disallow: /citydirs/1884/1884full.pdf
Disallow: /citydirs/1886/1886full.pdf
Disallow: /citydirs/1889/1889full.pdf
Disallow: /citydirs/1891/1891full.pdf
Disallow: /citydirs/1893/1893full.pdf=========
not allowed
http://www.oshkoshpubliclibrary.org/citydirs/1857/1857full.pdf
allowed http://www.oshkoshpubliclibrary.org/citydirs/1857/1857fullx.pdf

So, that works fine. But, if I spider with swish-e, it still doesn't
skip those files. Is there something else I'm missing?

Jody
Received on Fri Jun 27 13:11:57 2003