Using SWISH-E 2.4.2 on Solaris 5.8 i386
If, in my swish-e cfg I set:
obeyRobotsNoIndex yes
and I have the following in my content
<meta name="robots" content="noindex">
during indexing I see:
blah.html - Using DEFAULT (HTML2) parser - (Skipped due to Robots
Excluion Rule in meta tag)
Aside from the typo in the output, that's what I'd expect.
If I then add the following directive to my cfg:
IndexContents HTML .html
and rebuild the index
blah.html - Using HTML parser - (496 words)
Apologies if this is FAQ, but can't I get swish-e to obey the
"obeyRobotsNoIndex" rule primarily and only "IndexContents" if the meta
isn't present?
Thanks.
MattO
Received on Wed May 5 21:11:42 2004