Figured this out I think.
IndexContents HTML2 .html
Need to use the HTML2 parser to index the contents.
From: MattO [mailto:firstname.lastname@example.org]
Sent: Wednesday, May 05, 2004 9:05 PM
Subject: obeyRobotsNoIndex & IndexContents
Using SWISH-E 2.4.2 on Solaris 5.8 i386
If, in my swish-e cfg I set:
and I have the following in my content
<meta name="robots" content="noindex">
during indexing I see:
blah.html - Using DEFAULT (HTML2) parser - (Skipped due to Robots
Excluion Rule in meta tag)
Aside from the typo in the output, that's what I'd expect.
If I then add the following directive to my cfg:
IndexContents HTML .html
and rebuild the index
blah.html - Using HTML parser - (496 words)
Apologies if this is FAQ, but can't I get swish-e to obey the
"obeyRobotsNoIndex" rule primarily and only "IndexContents" if the meta
Received on Wed May 5 21:25:56 2004