On Mon, Feb 06, 2006 at 11:30:56PM -0600, dennis lastor wrote:
> When run the spider I get the following error(s):
> 
> Summary for: http://Internal_Site/WebHome
> Connection: Close: 1  (1.0/sec)
>       Unique URLs: 1  (1.0/sec)
>        robots.txt: 1  (1.0/sec)

robots.txt is telling it not to index that page.


-- 
Bill Moseley
moseley@hank.org

Unsubscribe from or help with the swish-e list: 
   http://swish-e.org/Discussion/

Help with Swish-e:
   http://swish-e.org/current/docs
   swish-e@sunsite.berkeley.edu