Skip to main content.
home | support | download

Back to List Archive

Re: noindex overwrite default

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Fri Mar 04 2005 - 18:03:30 GMT
On Fri, Mar 04, 2005 at 08:34:03AM -0800, Chris Martin wrote:
> How would you recommend implementing the filtering as I need to add tags
> at the top and bottom of my page and change existing ones as current the
> page is marked for where to search rather than where not to search.
> 
> I have tried changing spider.pl. Changing make_request seemed to be the
> correct place but I guess not because it doesn't work.

How about using a filter_content callback in your spider config?

Completely untested, but something like:

    filter_content => sub {
        my ( $uri, $server, $response, $text ) = @_;

        return 1 unless $response->content_type eq 'text/html';

        $$text =~ s/<body>/<body><!-- noindex -->/i;
        $$text =~ s/mysearchend/noindex/g;
        $$text =~ s/mysearchstart/index/g;

        return 1;
    },


Test it to make sure it work, of course.

Is your site generated from templates?  It would be helpful if your
index/noindex tags were in a template where changing would update the
entire site.

-- 
Bill Moseley
moseley@hank.org

Unsubscribe from or help with the swish-e list: 
   http://swish-e.org/Discussion/

Help with Swish-e:
   http://swish-e.org/current/docs
   swish-e@sunsite.berkeley.edu
Received on Fri Mar 4 10:03:32 2005