Skip to main content.
home | support | download

Back to List Archive

Increase COALESCE_BUFFER_MAX_SIZE in config.h

From: Steve Magee <smagee(at)not-real.arb.ca.gov>
Date: Tue Nov 15 2005 - 00:41:31 GMT
On our large website, I generate many indexes for the swish-e search 
engine to use to obtain results.  While generating one of the many 
indexes, I am now getting the following error...

err: Buffer too short in coalesce_word_locations. Increase 
COALESCE_BUFFER_MAX_SIZE in config.h and rebuild.

I'm on a Linux OS Kernel 2.6.9-22 with Swish-e 2.1-dev-24.

The error indicates either a binary file is being parsed or simply "too 
many words".  The directory that this one index is using has over 19,000 
documents.  If I remove several subdirectories (over 5,000 documents), 
the index is generated (12 MB), but not as large as the last know 
working index (14 MB) before I noticed this problem.

There is a binary file that is being parsed.  I have had no problems 
with this binary file being there in the past years.  My guess is that I 
have "too many words".  Users are constanctly adding and modifying 
files.  A much smaller directory that only has a bit over 5,000 
documents, generates an index of over 32 MB.  I have no problem with 
this directory or index.

I have gone into the ./src/config.h file and bumped up the value of 256 
MB to 1024 for COALESCE_BUFFER_MAX_SIZE, recompiled and I still get the 
same error.  The comments state that "COALESCE_BUFFER_MAX_SIZE should 
not be altered.

Besides breaking up the large directory into 2 and having 2 indexes, is 
there another option?  Breaking up the directory would cause a lot of 
work to recode the html and other files.

Thanks in advance,
Steve
Received on Mon Nov 14 16:41:31 2005