Skip to main content.
home | support | download

Back to List Archive

Re: Crawling

From: Bill Moseley <moseley(at)>
Date: Thu Nov 18 2004 - 14:39:38 GMT
On Thu, Nov 18, 2004 at 06:28:29AM -0800, wrote:
> Hi,I 'm studing a open source search engines  and I have a question : can
> swish-e  execute a distributed crawling ? For example on a set of  Pc in
> a Intranet. 

It's not designed for that.  There's no way for the spiders to
currently share a list of visited URLs.  I suspect it would be
reasonably easy for you to add that function.

But, if you can tell the spiders to index separate hosts they you
could concat their output for indexing.

Bill Moseley

Unsubscribe from or help with the swish-e list:

Help with Swish-e:
Received on Thu Nov 18 06:39:39 2004