Skip to main content.
home | support | download

Back to List Archive

Re: swish++ (was re: various messages on swish-e list)

From: Mark Gaulin <gaulin(at)not-real.globalspec.com>
Date: Mon May 17 1999 - 16:51:18 GMT
Hi Lars
Given where we are right now with swishe I can only suggest that you break
the indexing task up into smaller chunks (creating multiple indexes) and
merge them all into one big index when you are done. 

	Mark

At 08:32 AM 5/17/99 -0700, Lars Kellogg-Stedman wrote:
>Howdy,
>
>I've seen you frequent posts to the swish-e mailing lists along these
>lines:
>
>  > Switch to SWISH++.  It's far more efficient and faster.
>
>If you follow the list at all, you'll note that I've recently run into
>problems with excessive memory consumption by swish-e.  A faster and more
>efficient indexer would certainly be an attractive alternate.
>
>I just gave swish++ a try, and while it indeed may be faster, it still
>happily consumes memory until it slows by 128M O2 to a crawl due to
>swapping.  Trying to limit the amount of memory available to swish++ (via
>the ulimit shell command) simply causes it to die when the memory is
exhausted.
>
>I'm planning on moving the indexing operations to a spare Linux box.
>Unfortunately, this machine has only 32M; given the performance I'm seeing
>with 128M, I'm not sure what's going to happen with a much smaller amount
>of available memory.
>
>Is there any way to set an upper bound on the memory used by swish++
>without limiting the size of the data set that can be indexed?  A reduction
>in indexing speed is acceptable.
>
>Thanks,
>
>  -- Lars
>
>-- 
>Lars Kellogg-Stedman <lars@larsshack.org>
> 
Received on Mon May 17 09:47:27 1999