Skip to main content.
home | support | download

Back to List Archive

Re: DOC-Properties

From: Mark Gaulin <gaulin(at)not-real.globalspec.com>
Date: Tue May 11 1999 - 15:12:23 GMT
Q: In your config file do you specify any PropertyNames values?
If Yes, then be aware that all META tag contents matching those tag names
will be stored in the index. 
In No, then that is strange because the extra overhead in the index file is
not
large (a few bytes per file) when there are no PropertyNames specified.

Swish builds it's indexes in memory so it is not the best at handling very
large
indexing situations.  Since DocProperties are generally text strings it's
easy to
imagine that situation getting out of hand. Right now there is no support for
storing some of the index on disk during the indexing process. It sounds like
that should be addressed. (We could store file info (filename, title, size,
and
doc properties) in a large temporary sequential data file and store the seek 
position in memory rather than all of that data.)

In the mean time check your config file for the PropertyNames option and
see if it might conceivably require that much storage for your tags. Try
indexing
with that option commented out (rather than recompiling swish). 

A workable solution might be to create a number of smaller indexes and merge
them.  The index file will still be large but the process should be more
manageable.

	Mark

At 07:29 AM 5/11/99 -0700, Rainer Scherg wrote:
>Yek - I was a little astonished:
> 
> Swish-e Index-DB
>  with "#define SUPPORT_DOC_PROPERTIES 1":  192 MB
>  with SUPPORT_DOC_PROPERTIES not set:       24 MB.
>
> We were not able to index 700 MB with SUPPORT_DOC_PROPERTIES over
>night...
> (Sparc Enterprise Server, 4 CPUs, lots of memory...)
>
> Am I doing something wrong?
>
>
>-- Rainer
>
>-- 
>--------------------------------------------------------------------
>Rainer Scherg,  RTC,  Mannesmann Rexroth AG,  http://www.rexroth.com
>e-Mail: Rainer.Scherg@rexroth.de,   Tel/Fax: +49-9352-18-1510 /-1500
> 
Received on Tue May 11 08:10:24 1999