Skip to main content.
home | support | download

Back to List Archive

Re: size of database

From: Craig A Summerhill <craig(at)not-real.cni.org>
Date: Tue Feb 03 1998 - 12:19:38 GMT
On Tue, 3 Feb 1998, Jean-Louis Maltret <jlm@eiffel.univ-mrs.fr> wrote:
> 
> I have questions about size of base indexed by swish-e . Could
> someone answer the reasonable size (number of files and total
> size) in function of machine used ? What is the maximal example
> among users of swish-e ?
> 
> When I was user of wais I'd heard of 100Mbytes databases indexed
> ans served by wais on ordinary workstations. Is there some example
> like this ?

One of our websites has about 750 Megabytes indexed with swish-e.  That
is raw data (including HTML tags, meta tags, and comments some of which
are indexed) Of course, I have never attempted to index the entire site
as one job.  I have it broken into logical pieces which each get indexed
on their own, and then once a week I run a job which merges the pieces
into a master index.  My largest individual index file is about 20 Meg,
but I've got a half dozen others that are 10-12 Meg.  The combined index 
is around 75 Meg.

I'm afraid that swish-e would choke the machine (for RAM) if I tried to
index it all at once.  I'm finding it to be a real memory hog...
-- 

   Craig A. Summerhill, Systems Coordinator and Program Officer
   Coalition for Networked Information
   21 Dupont Circle, N.W., Washington, D.C.   20036
   Internet: craig@cni.org   AT&Tnet (202) 296-5098
Received on Tue Feb 3 04:30:46 1998