Skip to main content.
home | support | download

Back to List Archive

Indexing Off Site Links

From: Antonio Barrera <abarrera(at)not-real.Princeton.EDU>
Date: Thu Sep 16 2004 - 19:07:44 GMT
I've seen some threads about similar problems to the one I'm facing, yet
many were older solutions.

My base url is: http://library.princeton.edu .  However, there are links to
other servers which I would want to index, without indexing the entire site.
Prior to indexing I have some knowledge of servers/directories, I do want to
search.  

For instance:  I may want to index,
http://www.princeton.edu/~rbsc/exhibitions/online.html but not all of
www.princeton.edu.  Or I may want to do
http://libweb5.princeton.edu/ejournals/by_title_zd.asp but not all of
libweb5.princeton.edu.

Any thoughts or ideas?  I'm using spider.pl with some configuration
directives.

TIA,
Antonio Barrera
Library Web Development Manager
Princeton University
Received on Thu Sep 16 12:09:18 2004