Skip to main content.
home | support | download

Back to List Archive

Duplicate Documents and Digest::MD5

From: Jon Sorensen <jon(at)not-real.starkmedia.com>
Date: Fri Oct 01 2004 - 14:36:22 GMT
Duplicate Documents

in http://swish-e.org/current/docs/spider.html says to use

Digest::MD5 to fingerprint indexed pages and prevent indexing duplicates
pages

Do I *just* need to install the module for this to work? I'm not  clear on
this

thanks for any help. Thanks for pointing out the SSL support in the docs
too.
I can index https pages now.

thanks again

Jon Sorensen
Developer

STARKMEDIA | interactive solutions
219 N. Milwaukee Street
Milwaukee, WI 53202

p 414.226.2710
f 414.226.2716
e jon@starkmedia.com
Received on Fri Oct 1 07:38:27 2004