On Fri, Feb 20, 2004 at 01:31:48AM -0800, Ander wrote:
> Hi all,
>
> I want to use the test response callback to make an error control and insert into a database the broken links or URL failures. But I have realised, that for example, when I find a 404 error (page not found) the spider didn't execute test_response's callback funtions.
>
> What can I do??
Edit the source of spider.pl. You might be able to do something in this
section:
# Report bad links (excluding those skipped by robots.txt
if ( $server->{validate_links} && !$response->is_success ) {
validate_link( $server, $uri, $parent, $response );
}
>
--
Bill Moseley
moseley@hank.org
Received on Fri Feb 20 06:23:35 2004