What are the advantages and disadvantages of indexing via the the 
spider?

I want to index the content of many electronic serials, specifically, 
all or part of the serials listed the Directory of Open Access 
Journals:

   http://www.doaj.org/

I suppose I could use spider.pl to crawl the remote files and index 
them. I could also use something like wget to create mirrors of the 
files and index them that way.

What are the advantages and disadvantages of either approach? If I use 
the spider, the I don't need nearly as much local disk space. If I do 
the mirroring thing, then I have local copies and I save on network 
bandwidth.

-- 
Eric Lease Morgan
Head, Digital Access and Information Architecture Department
University Libraries of Notre Dame

(574) 631-8604