On Fri, Mar 04, 2005 at 08:34:03AM -0800, Chris Martin wrote:
> How would you recommend implementing the filtering as I need to add tags
> at the top and bottom of my page and change existing ones as current the
> page is marked for where to search rather than where not to search.
>
> I have tried changing spider.pl. Changing make_request seemed to be the
> correct place but I guess not because it doesn't work.
How about using a filter_content callback in your spider config?
Completely untested, but something like:
filter_content => sub {
my ( $uri, $server, $response, $text ) = @_;
return 1 unless $response->content_type eq 'text/html';
$$text =~ s/<body>/<body><!-- noindex -->/i;
$$text =~ s/mysearchend/noindex/g;
$$text =~ s/mysearchstart/index/g;
return 1;
},
Test it to make sure it work, of course.
Is your site generated from templates? It would be helpful if your
index/noindex tags were in a template where changing would update the
entire site.
--
Bill Moseley
moseley@hank.org
Unsubscribe from or help with the swish-e list:
http://swish-e.org/Discussion/
Help with Swish-e:
http://swish-e.org/current/docs
swish-e@sunsite.berkeley.edu
Received on Fri Mar 4 10:03:32 2005