Skip to main content.
home | support | download

Back to List Archive

Re: [swish-e] Would you recommend swish-e in this scenario?

From: Judith Retief <JudithR(at)not-real.inet.co.za>
Date: Fri Oct 01 2010 - 10:35:16 GMT
> I wouldn't recommend searching and indexing simultaneously anyway :-)

Just as an aside, we search and index simultaneously without a problem. We manage this by searching and indexing into seperate files, merging at intervals.


Searches are done against two index files: a master and a daily.

We get content arriving throughout the day (sometimes a few items per second), and have a way of knowing which content is newly arrived and yet has to be indexed. We index those throughout the day, in batches of about 1000, indexing into a seperate, temporary index file. Once that index is done, we merge it with a daily index. 

Before the merge, we first copy the daily index file, and merge against the copy; this means the daily is still available for searches. 

When the merge is done, we rename copy to that of the daily. Searches are only unvailable for the very short time that the file is renamed.

At day-end, the daily is merged into a master. 
_______________________________________________
Users mailing list
Users@lists.swish-e.org
http://lists.swish-e.org/listinfo/users
Received on Fri Oct 1 06:36:46 2010