Skip to main content.
home | support | download

Back to List Archive

Re: suggestions for large multi-server indexing?

From: Ron Samuel Klatchko <rsk(at)not-real.corpmail.brightmail.com>
Date: Fri Jun 09 2000 - 23:41:49 GMT
Albert Lunde wrote:
> (1) What are the are the pros and cons of doing a single big index
> (giving it starting URLs across all servers) vs. doing a number of
> small indexes and merging them?

The nice thing about doing several small indexes is that you can do them
in parallel and get things done quicker.  It's also easier to do
maintenance; if you know some sites change often and others don't; you
can reindex those that do on a regular basis and just reuse the indexes
from those that don't.  Finally, it's a more robust solution.  If one of
the servers is down, you can still get things going.

moo
------------------------------------------------------------
        Ron Samuel Klatchko - Senior Software Jester
            Brightmail Inc - rsk@brightmail.com
Received on Fri Jun 9 19:44:26 2000