Well I started using the spider.pl because the swishspider just didn't seem
to be doing the trick.  Any idea why use_md5 is failing me?  I have two urls
that point to the same page:

http://www.wslife.com/operator.asp?location=home&location=planning+and+retir
ement
http://www.wslife.com/operator.asp?location=home&location=planning%20and%20r
etirement

I have set use_md5 to true, but both of these links come up in search
results.  My config is below.

config:

@servers = (
 {
 base_url    => 'http://144.10.10.56:8093/dev/testxslt.asp',
 same_hosts  => [ qw/swish-e.org/ ],
 email       => 'tom.mcdonald@wslife.com',
 delay_min => .025,
 use_md5  => 1,
 max_depth => 1
 # other spider settings described below
 },
);
Thomas McDonald
Title: Principal Consultant
Sogeti USA
4445 Lake Forest Drive
Suite 550
Cincinnati, OH 45242

Office: (513) 563-6622
Mobile: (513) 257-7281
Fax: (801)340-9083
E-mail: thomas.mcdonald@sogeti-usa.com







*********************************************************************
Due to deletion of content types excluded from this list by policy,
this multipart message was reduced to a single part, and from there
to a plain text message.
*********************************************************************