Skip to main content.
home | support | download

Back to List Archive

RE: size limitations or...

From: Nicolas Huillard <nhuillard(at)>
Date: Fri Jul 09 1999 - 11:36:20 GMT
I have the same problem, but I list all files to index in the config file. 
When there is more than 4000 files in the config file, this error occurs.  I didn't test heavily this cases, but I think swish-e reads all filenames, and stores them, before indexing. If there is too much filenames, it stops, without overwriting the preceding index file.
The solution (for me, as I list all files in the config file) is to create multiple sub-indexes, and merge them. I don't know how you could handle you case, if the problem is the same. Maybe create an index for each subdirectory of your HTML tree, and then merge them. Or use the spider that retreives each file one by one...


Nicolas Huillard
Directeur Technique
TÚl : +33 1 43 21 16 66
Fax : +33 1 56 54 02 18

-----Message d'origine-----
De:	Simon Baumberg []
Date:	vendredi 9 juillet 1999 12:53
└:	Multiple recipients of list
Objet:	[SWISH-E] size limitations or...


I have begun to configure and use Swish-e on our intranet for the first time
and have run into the following error message while trying to create an
index file. 

/usr/local/bin/swish-e -i /mnt/htmldocs/ -c /opt/swish/src/user.config -f
./search/swish/index -v 3 > swishindex.log
swish: Ran out of memory (could not allocate enough)!

I have roughly over 20,000 files (including .gifs etc, although I have opted
not to index them in the config file) using roughly 330MB space on the hd. I
have roughly 310MB free space on the hd.

server harddisk
Used : ~330 MB
Available : ~310
Total : ~640 MB

Has anybody experienced problems with similar amounts of files to index?
Since I have no idea how to proceed with this problem except blindly testing
different configurations and go from there, I would appreciate if somebody
with more experience with swish-e could point me in the general direction of
the problem/solution.

Thanks in advance
S. Baumberg
Received on Fri Jul 9 04:32:56 1999