Hi all,

I'm having trouble narrowing my index down in a few key areas... basically I have an ASP page that contains a number of different querystrings, some of which I want indexed and some I don't want indexed.
I'm pretty sure that it's my sucky PERL coding that's causing the problems so I just wanted to run it past the forum..

The querystrings that I wish to run through the 'no_index' callback function all start with "www.planetpdf.com/tools.asp?webpageid=615&SearchType=Product". There are many different endings to this string (heaps of different 'Product' keywords) but any querystring like this I only want spidered and not indexed.

DETAILS:

--- START Conf file ---

IndexFile c:\swish-e\toolsIndex.index
IndexDir spider.pl
IndexReport 3
IgnoreMetaTags script
obeyRobotsNoIndex yes
StoreDescription HTML2 <description> 500
SwishProgParameters toolsSpider.config

--- END Conf file ---

--- START Spider Config file, toolsSpider.config ---
@servers = (
     {
         debug =>DEBUG_SKIPPED | DEBUG_ERRORS,
         base_url    => 'http://cm3.planetpdf.com/tools.asp',
	 email => 'binary@binarything.com',
	 agent => 'pp_tools',
        keep_alive=>1,        
        test_response => sub {
        my $server = $_[1];
	#try and not index tools.asp?webpageid=615&SearchType=Product..	
      $server->{no_index}++ if $_[0]->path =~ /tools\.asp\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->path =~ /\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->query_form =~ /\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->query_form =~ /tools\.asp\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->path_query =~ /\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->path_query =~ /tools\.asp\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->query =~ /\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->query =~ /tools\.asp\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->query_keywords=~ /\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->query_keywords=~ /tools\.asp\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->path_segments=~ /\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->path_segments=~ /tools\.asp\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->path_query=~ /\?webpageid\=615\&SearchType\=Product$/;
	$server->{no_index}++ if $_[0]->path_query=~ /tools\.asp\?webpageid\=615\&SearchType\=Product$/;
	return 1;
	},
	use_md5 => 1, 
     },
 );

--- END Spider Config file ---

--- Run Command ---
C:\SWISH-E>swish-e -S prog -c PPTools.conf

--- 

As you can see, I've played around with a variety of different URI functions, and none of them seem to be able to stop that URL from being indexed. The query_form, query and query_keywords calls all produce a "Use of uninitialised value in pattern match (m//)at line blah" warning too.

Any pointers?

Tim