Skip to main content.
home | support | download

Back to List Archive

Re: a better way to index?

From: Hup Chen <hup(at)not-real.addall.com>
Date: Mon Mar 10 2003 - 19:43:59 GMT
> > Method 1:
> > index the whole 2M database
> > # $parse $source | $swishe -i stdin -S prog -f $pathOut/$test1 -c
> > swish.conf
>
> What was the max memory usage?


  The external parse program was using 2GB, swish-e took about 750MB RAM.


> > Method 1: took 3 hours 13 minutes
>
> Can you run that again with -e and report the time and max memory usage.
>
> As the hash tables fill indexing slows down.

  under -e mode, swish-e used only up to 150MB, but the external program
was still using 2GB.  The index time was about the same, 3:15.

Elapsed time: 03:15:27 CPU time: 00:-4:-3


> Nice to see merge is working fast.  It was not that long ago that it was
> very slow and memory intensive.

  I am glad to find swish-e.  Great search performance, reasonable index
time, cool price (some commercial product quoted me $50K-100K/per CPU).

> BTW -- nice hardware. ;)
  Thanks :)


--
Hup
Technical Manager
AddALL.com

Search.  Compare.  Save.

http://www.addall.com  (new book search)
http://used.addall.com/  (used book search)
http://www.amusicarea.com  (CDs)
http://www.amoviearea.com  (DVD & VHS)
http://www.amagarea.com  (magazine subscriptions)
Received on Mon Mar 10 19:50:14 2003