Mark Gaulin wrote:

> Allowing the person who configures swish to specify *some* file
> extensions that he does not want indexed/spidered is just feature.

Thanks ! At least someone who understand what is an "option"...
and why we call it like that...  :)

> We all know that some of the big search engines (altavista, infoseek,
> etc, etc) do not try to index pages that have a certain "look" to them...
> some skip "cgi" or ".exe", others anything with a "?" or "&" in them.
> I think I could say fairly certainly that AltaVista is not trying to download
> any of the gif files from my site.  It does not know *for sure* if those

That's right. Actualy, proxy/cache have a cache stop-list that
usualy ignore any URL containing a "/cgi-bin/" or a "?" (search
string). It's also a _configurable option_

> Having said all of that, I am not saying that someone specific "must go
> implement this right now, or else!"...  I'm just saying that this feature is
> not wrong or a sign of stupidity, and in some cases, is highly desirable.

Actualy it's already implemented : it's in the filesystem method. It
was just not included in the HTTP method source file. A simply
cut-and-copy is nearly enough to add it back. I did it and posted
the new file sometime ago...
 
Cheers,
Yann Stettler

-- 
-------------------------------------------------------------------
TheNet - Internet Services AG              CohProg SaRL
stettler@thenet.ch                         stettler@cohprog.com
http://www.thenet.ch/                      http://www.cohprog.com/
                              ---**---
Anime and Manga Services                   http://www.animanga.com/