Skip to main content.
home | support | download

Back to List Archive

Re: [swish-e] trading off nr vs size of index files

From: Judith Retief <JudithR(at)not-real.inet.co.za>
Date: Thu Feb 21 2008 - 08:18:02 GMT
> I did some testing a few years ago and (if memory serves) 
> found that searching on two indexes was about 10-20% slower
> than searching on the same data in one index.

Thanks very much. Merging into a smaller file is much much faster than merging into a large one, so we broke our index files up into about 6 seperate files and merge in parallel. We haven't noticed an influence on the searches because I guess the 10-20% performance hit per file wasn't noticable as the searches were pretty fast to start with (and we only need to search through the 2/3 files that the user is interested in)

But we're considering splitting up into perhaps 100 files - and with all our added content all those 10%'s are going to catch up on us. Will have to think of something else.
_______________________________________________
Users mailing list
Users@lists.swish-e.org
http://lists.swish-e.org/listinfo/users
Received on Thu Feb 21 03:18:06 2008