Skip to main content.
home | support | download

Back to List Archive

SV: RE: Indexing large nbrs of docs

From: Bruusgaard Jan <jan.bruusgaard(at)not-real.ssb.no>
Date: Fri Jun 01 2001 - 12:43:35 GMT
I tried to index 49194 mainly html and txt documents on a new Linux-box with
512k physical and 512k swap memory.

The indexing process took more than 4 hours with swish-e 2.05

I now installed 2.1-dev 21 

The figures now were:
 17 min 41 sec with -e switch
 19 minutes, 46 seconds without -e


Jan Bruusgaard


> ----------
> Fra: 	Rainer.Scherg@rexroth.de[SMTP:Rainer.Scherg@rexroth.de]
> Svar til: 	Rainer.Scherg@rexroth.de
> Sendt: 	1. juni 2001 12:43
> Til: 	Multiple recipients of list
> Emne: 	[SWISH-E] RE: Indexing large nbrs of docs
> 
> use e.g. "top" on Solaris to check the usage of memory.
> 
> You should see to amount of used memory by swish and also
> the free real memory and swap space.
> 
> You can also try the "-e" swish option to use fewer memory.
> 
> cu - rainer
> 
> 
> > -----Original Message-----
> > From: Greg Caulton [mailto:gcaulton@sympatico.ca]
> > Sent: Friday, June 01, 2001 4:43 AM
> > To: Multiple recipients of list
> > Subject: [SWISH-E] Indexing large nbrs of docs
> > 
> > 
> > Hi,
> > 
> >     Large, well compared to my other indexes :-)
> > 
> >     I wish to index a directory with 2800 word docs, of which 
> > the total
> > combined size is 720MB.
> > 
> >     However the indexing is getting slower and slower as the number of
> > documents indexed increases - and I believe it will run for several
> > hours before slowing to a crawl.
> >   
> >     This is version 2.0.  Is there are a more recent version 
> > that might
> > relieve this problem?
> > 
> >     Is it possible to merge seperate smaller indexes?
> > 
> >     I am running this on Solaris with only 256MB real memory.
> > 
> > thanks!
> > 
> > Greg
> > 
> > 
> > -----------------------------------------------------------
> > This Mail has been checked for Viruses
> > Attention: Encrypted Mails can NOT be checked !
> > 
> > ***
> > 
> > Diese Mail wurde auf Viren ueberprueft
> > Hinweis: Verschluesselte Mails koennen NICHT geprueft werden!
> > ------------------------------------------------------------
> > 
> 
Received on Fri Jun 1 12:44:55 2001