Skip to main content.
home | support | download

Back to List Archive

Re: Incremental updating

From: Peter Karman <karman(at)not-real.cray.com>
Date: Thu Jun 10 2004 - 15:59:53 GMT
How timely. I've just started doing this. These are XML files indexed 
via a perl -S prog filter.

pubs@icarus 109% swish-e -M active.index maint.index fixes.index 
maninfo.index
Input index 'active.index' has 8064 files and 26952 words
Input index 'maint.index' has 5441 files and 18375 words
Input index 'fixes.index' has 4740 files and 17380 words
Getting words in index 'active.index':  26952 words
Getting words in index 'maint.index':  18375 words
Getting words in index 'fixes.index':  17380 words
Processing words in index 'maninfo.index':  42445 words
Removed      0 words no longer present in docs for index 'maninfo.index'
Writing main index...
Sorting words ...
Sorting 42,445 words alphabetically
Writing header ...
Writing index entries ...
   Writing word text: Complete
   Writing word hash: Complete
   Writing word data: Complete
42,445 unique words indexed.
24 properties sorted.
18,245 files indexed.  0 total bytes.  1,274,441 total words.
Elapsed time: 00:01:23 CPU time: 00:01:22
Indexing done!

pubs@icarus 111% uname -a
SunOS icarus.us.cray.com 5.8 Generic_108528-19 sun4u sparc 
SUNW,Sun-Blade-100


Bill Moseley wrote on 06/10/2004 09:57 AM:

> Could you post some comments about the number of indexes, number of
> files in each, type of files and their average size and the time it
> takes to index and merge?  Perhaps trimmed down output from indexing.
> 
> People often ask about this so it would be nice to have this as an
> example.
> 
> 
> 

-- 
Peter Karman - Software Publications Programmer - Cray Inc
phone: 651-605-9009 - mailto:karman@cray.com
Received on Thu Jun 10 15:59:59 2004