At 12:52 PM 07/31/02 -0700, Khalid Shukri wrote:
>> Luckily, you are running linux so you can use lsof to see what files the
>> process is indeed holding open.
>
>Seem to be mostly TCP sockets. However, I reach a peak of about 17000 open
>handles. Wonder what's happening there!

What state are they in?  What does netstat show?  

You might also make sure you are using a very current version of LWP -- and
then try and use the KeepAlive feature, although I think you said you were
not fetching that many pages from each server.

For your situation using a spider that fetches URLs in parallel might be a
good thing.


-- 
Bill Moseley
mailto:moseley@hank.org