Skip to main content.
home | support | download

Back to List Archive

Re: [swish-e] How do I index via HTTP when authentication is

From: Adam Douglas <ADouglas(at)not-real.venmarces.com>
Date: Wed Feb 20 2008 - 22:29:35 GMT
After some searching with Google for "500 Can't locate object method
"new" via package "LWP::Protocol::https::Socket" I discovered that LWP
does not support SSL without a Perl module. It is my understanding that
Crypt::SSLeay is recommended to be used. I tried to install it but had
no success. So for now I just removed HTTPS and used HTTP in the
SwishSpiderConfig.pl just to get this puppy working finally. Can go back
and figure out installing Crypt::SSLeay later. 

So now when I run "swish-e -S prog -c swishe.venmarces.private.conf" I
receive the following message below. It appears to be trying to login
but still something is wrong as it does not index anything nor appear to
be logged in. I checked my sessions and last login date for the account
I'm using and no successful login. Do I have something wrong with the
test_url() ?

Indexing Data Source: "External-Program"
Indexing "/usr/local/lib/swish-e/spider.pl"
External Program found: /usr/local/lib/swish-e/spider.pl
/usr/local/lib/swish-e/spider.pl: Reading parameters from
'SwishSpiderConfig.pl'

 -- Starting to spider:
http://blowfish.venmarces.com/login/?szID=username&szPWD=password --
?Testing 'test_url' user supplied function #1
'http://blowfish.venmarces.com/login/?szID=username&szPWD=password'
+Passed all 1 tests for 'test_url' user supplied function
RobotRules <http://blowfish.venmarces.com/robots.txt>: Unexpected line:
Sitemap: http://www.venmarces.com/sitemap.xml

vvvvvvvvvvvvvvvv HEADERS for
http://blowfish.venmarces.com/login/?szID=username&szPWD=password
vvvvvvvvvvvvvvvvvvvvv

---- Request ------
GET http://blowfish.venmarces.com/login/?szID=username&szPWD=password
Accept-Encoding: gzip, x-gzip, deflate
From: webmaster@venmarces.com
User-Agent: swish-e spider http://swish-e.org/

---- Response ---
Status: 302 Found
blah blah blah removed the extra header details.

^^^^^^^^^^^^^^^ END HEADERS ^^^^^^^^^^^^^^^^^^^^^^^^^^
Summary for:
http://blowfish.venmarces.com/login/?szID=username&szPWD=password
Connection: Close: 1  (0.3/sec)
   Off-site links: 1  (0.3/sec)
      Unique URLs: 1  (0.3/sec)

Removing very common words...
no words removed.
Writing main index...
err: No unique words indexed!

> ---- Response ---
> Status: 500 Can't locate object method "new" via package 
> "LWP::Protocol::https::Socket"
> Content-Type: text/plain
> Client-Date: Wed, 20 Feb 2008 20:11:47 GMT
> Client-Warning: Internal response
> 
> ^^^^^^^^^^^^^^^ END HEADERS ^^^^^^^^^^^^^^^^^^^^^^^^^^

This message (including any attachments) is intended only for the use of the individual or entity to which it is addressed and may contain information that is non-public, proprietary,privileged, confidential, and exempt from disclosure under applicable law or may constitute as attorney work product. If you are not the intended recipient, you are hereby notified that any use, dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, notify us immediately by telephone and
(i) destroy this message if a facsimile or (ii) delete this message
immediately if this is an electronic communication. Thank you.
_______________________________________________
Users mailing list
Users@lists.swish-e.org
http://lists.swish-e.org/listinfo/users
Received on Wed Feb 20 17:29:35 2008