I'm trying to get swish-e working on our internal web server box (working
great for years on our exernal web server, I love it!). It's running Solaris
2.6 (yes, ancient).
I initially did the latest stable version (2.4.2), but am now trying with
2.5.2.
With 2.4.2, no -e flag:
The initial issue, when indexing the files, is that swish-e is using up gobs
and gobs of memory until it crashes and/or causes the various other
processes to crash. The error message is "unable to allocate additional X
bytes". I can watch (with "top") as the memory usage grows and grows during
the indexing process.
The biggest chunk to index (/home/manuals) is mostly .pdf files, and is
about 15GB in size. In fact, with this testing, I am only trying to index
this chunk of the filesystem.
I tried the -e flag, after seeing some suggestions to do so. This gives me
this error, after indexing less than 100 files:
err: Couldn't create temporary file './swtmploc7wZ8j_' file descriptor: No
such file or directory
Furthermore, in the directory where I started swish (and NOT in any tmp
dir), it is filled with 252 files named swtmplocXXXXX_, where XXXXX seems to
be random characters (a hash of some sort?). I tried bumping up the file
descriptor limit to 9999 with the "ulimit" command, to no apparent effect.
Should the -e flag be creating so many files? It's more than one per file
indexed, apparently!
I then tried with -e with the latest 2.5.2 (from today), and got the same
behaviour.
I am right now trying to re-index without the -e command, but I see already
that swish-e has consumed 230MB, and is rapidly growing.
For reference, here is what is in the swish.conf file (I commented out
everything but the big /home/manuals section):
###########################################################
IndexOnly .html .txt .pdf .htm .doc
NoContents .gif .jpg
#
# Don't do these pdf's - crashing
FileRules pathname contains marketing/competition
#
# Index the main internal section
#
#IndexDir /www/internal
ReplaceRules remove /www/internal
#
# Index the internal_web section on titan
#
#IndexDir /home/groups/internal_web
ReplaceRules remove /home/groups/internal_web
#
# Index the manuals section
#
IndexDir /home/manuals
ReplaceRules remove /home
###########################################################
And here is how I am invoking swish-e:
/bin/nice -n 19 /usr/local/bin/swish-e -c /usr/local/swish/swish.conf -f
/usr/local/swish/index.swish-e
Any suggestions?
thanks!
johnS
Received on Thu Dec 2 15:17:26 2004