Bill Moseley wrote:
> It can get confusing since there's so many ways to do things and since
> different programs are doing different parts of the indexing.
Yes, I noticed that. :-)
I'm starting to get the idea of how it works.
> spider.pl default http://yoursite.to.index/ > out.txt
Thanks, I hadn't read far enough to know about that "default" option. I was
busy setting up a config file based on the minimal example - if I'd seen
that line in the docs first I would have done that straight away.
So, anyway, I did that and got the main index page, so I know it works.
My mission is to allow searching in some password-protected sub-sites that
aren't linked from the main page so I think I'll have to do them each
Would it make sense to maintain a separate index for each one rather than
put it all in together with the main index, even though they're all pretty
I think I like the idea of leaving the main site index as it is and treating
the new bits separately.
(Thinking out loud...)
Received on Wed Dec 6 19:17:20 2006