On Tue, Nov 25, 2003 at 01:45:05PM -0800, Kissman, Paul (BLC) wrote:
> I am using the Swish-E (version 2.4.0) prog method along with the
> supplied spider script and the Filters Option to index both html and pdf
> files (and Word files) on our web site. I've got my SwishSpiderConfig
> file working; everything is fine in all regards but one. Most of my
> website's html pages use server side includes, and I am not getting any
> lastmodified date information for these shtml files.
look for xbithack
> After some digging around I find out that the LWP package can't find the
> file modification date because the full page is generated dynamically
> through http and the filesystem modification timestamp for the main part
> of the page is not available to it.
Kind of. See above.
> I was thinking that one could insert a function in spider.pl that would
> quickly map the URL to the actual file, then go out and grab the actual
> file's timestamp if the web page were on the local server, and then
> stuff it in as the "Last-Mtime" value in the $headers string that gets
> returned to the indexer.
Sure if your httpd.conf is simply a one-to-one mapping to DocumentRoot.
> Is this a reasonable approach? Has anyone done this or solved this
> problem a different way?
Well (for fun), you could try this, but I would not recommend it. Say your
DocumentRoot is /var/www
spider.pl default file:///var/www/index.html
Received on Tue Nov 25 22:23:59 2003