Skip to main content.
home | support | download

Back to List Archive

RE: spider a database

From: Michael Porcaro <music(at)not-real.recordhall.com>
Date: Sun Nov 06 2005 - 17:13:23 GMT
I was using the e-switch, I think.

Here is my swish.conf file:

IndexDir spider.pl
IndexOnly .html
SwishProgParameters default http://www.youngcomposers.com/
StoreDescription TXT* 10000
StoreDescription HTML* <body> 10000
FuzzyIndexingMode Stemming_en

Here is the command I typed:

Swish-e -S prog -c swish.conf

Is everything correct?  I was using a windows fsecure ssh client to do
the spidering by the way.  Maybe I should not use a windows ssh client
and just stick with a cron job?

-----Original Message-----
From: Bill Moseley [mailto:moseley@hank.org] 
Sent: Sunday, November 06, 2005 9:08 AM
To: Michael Porcaro
Cc: 'Multiple recipients of list'
Subject: Re: spider a database

On Sun, Nov 06, 2005 at 02:33:27AM -0500, Michael Porcaro wrote:
> After spidering my whole site for about 3 or 4 hours, I got this
error:
> 
> err: Property Compression Error.  zlib compress2 returned: -4  Prop
len:
> 106 compress buf size: 1107 compress level:-1

A -4 is a memory error.  I'm not exactly sure what memory error that
is, but maybe you used up all RAM while indexing.  Use the -e switch
while indexing to reduce memory usage.

-- 
Bill Moseley
moseley@hank.org

Unsubscribe from or help with the swish-e list: 
   http://swish-e.org/Discussion/

Help with Swish-e:
   http://swish-e.org/current/docs
   swish-e@sunsite.berkeley.edu
Received on Sun Nov 6 09:13:31 2005