> On Sun, 2002-07-14 at 22:35, Michael Robinton wrote:
> > It looks to me like the merge in 2.1 might be slower than 2.05. I'm
> > merging 30.2 megs and 1.3 megs and seeing about a 95 minute run time on
> > 500mhz celeron linux2.4x with 500megs of memory. Maximum merge size is
> > about 200 megs at the end.
> 
> Merge has been fairly well ignored in recent updates.  I think Bill
> pointed out a few weeks ago that it's actually much faster to
> reindex the entire site than to to merge several smaller index
> files.  I think merge might get more attention once incremental
> indexing is supported.
> 
> > 2.05 seemed to run faster? but took a lot more memory.
> 
> How much faster?  Under most conditions 2.1 is considerably faster
> than older versions.

umm.... only subjective opinion at this point, I've taken it off the 
system (production). The 2.1 indexes are clearly faster, but the 
merges are slower than 2.05 -- hard to say quantitatively now. My 
guess is around 2:1 for files as above at least. I used to be able to 
merge the whole thing in a day or two but based on the benchmark of 
above, it would take 80-100 hrs or more.

> 
> > Has there been a trade-off made for merging small vs large files??
> 
> Well, I haven't a clue.  It could be a side effect of merge not
> being efficient.  But, generally 2.1 does sacrifice some performance
> to reduce RAM and disk usage.  You might pose the question on the
> discussion list.  Jose or Bill would probably have a better idea.
> 
> The -e (economy) option in the current CVS is a good example where a
> little performance is sacrificed.  It will generate hundreds of temp
> files then combine them back into the index file.  Memory usage is
> very minimal.  And, it's not really much slower.
> 
> -- 
>  David Norris
>   Dave's Web - http://www.webaugur.com/dave/
>   Augury Net - http://augur.homeip.net/
>   ICQ - 412039
>