Skip to main content.
home | support | download

Back to List Archive

Re: swishspider multiple sites

From: Ron Samuel Klatchko <rsk(at)not-real.corpmail.brightmail.com>
Date: Mon Apr 17 2000 - 16:48:21 GMT
Benedict wrote:
> Is there a command to index multiple sites using spidering method ?
> 
> swish-e -f sites.swish-e -S http -i ?

Yes, but it depends on exactly what you want to do.   If you want to
index site A, site B and site C, just specify each url in the config
file with IndexDir:
  IndexDir http://www.sitea.com/
  IndexDir http://www.siteb.com/
  IndexDir http://www.sitec.com/
  etc.

If on the other hand, you want to start at Site A and then index each
site it references, that's a bit harder.  You can fake it using
EquivalentServers but you still need to figure out which ones to use. 
The problem here is that deciding when to stop following links is
difficult, there are many different ways people want to decide when to
stop and I could not think up a general solution that works well for all
the cases.

moo
------------------------------------------------------------
        Ron Samuel Klatchko - Senior Software Jester
            Brightmail Inc - rsk@brightmail.com
Received on Mon Apr 17 12:50:02 2000