Skip to main content.
home | support | download

Back to List Archive

Re: Error control form test response

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Fri Feb 20 2004 - 14:23:35 GMT
On Fri, Feb 20, 2004 at 01:31:48AM -0800, Ander wrote:
> Hi all,
> 
> I want to use the test response callback to make an error control and insert into a database the broken links or URL failures. But I have realised, that for example, when I find a 404 error (page not found) the spider didn't execute test_response's callback funtions.
> 
> What can I do??

Edit the source of spider.pl.  You might be able to do something in this
section:

    # Report bad links (excluding those skipped by robots.txt

    if ( $server->{validate_links} && !$response->is_success ) {
        validate_link( $server, $uri, $parent, $response );
    }



> 

-- 
Bill Moseley
moseley@hank.org
Received on Fri Feb 20 06:23:35 2004