At 08:28 AM 08/20/02 -0700, Clem McDonald wrote:
>My questions are:
> a) Does the index now contain a word distance from the start of a
No, it's more of a position counter -- and there's only one counter (i.e.
not one for each metaname). It's bumped for each word, then it's bumped
when a meta name if found (to prevent matching phrases across metanames),
and there's also a config option to bump it on a set of characters. This
allows doing something like:
<meta name="subjects" content="some subject|another thing">
and not matching on the phrase "subject another"
> b) Does the phrase search represent a kind of proximity search as I
(answering without looking at the code...)
Only in that it checks that a word's position in a given file is only one
more then the previous word. I don't see why you couldn't say within, say,
five word positions -- but it might not be exactly five word positions due
to things mentioned above and stop words.
> c) Is there now something in the works for defining proximity in terms
>of word counts?
I don't know. Is there now? ;)
I don't know how valuable proximity searching would be. I think I'd rather
see work on the ranking so that if you search for a few words (not a
phrase) that docs where those words are close together are ranked higher
than words that are not close together.
>So you could not tell the difference between the two cases when looking
>for Hello world.
> Document 1
> <tag a> Hello World < tag a/>
> <tag a> hello <tag a/>
> <tag a> world <tag /a>
Right, searching for the (non-phrase) "hello world" would find both.
>This is important to our application where we may have diagnoses made up
>of an anatomic part and and a pathology part e.g.
> <diagnosis> Prostate benign < diagnosis>
> < diagnosis > Colon cancer < diagnosis>
I'm not completely following.
<diagnosis> Prostate benign <diagnosis>
<diagnosis> Colon cancer <diagnosis>
> ./swish-e -c c -i 1.xml -T indexed_words -v0
Adding:[1:diagnosis(10)] 'prostate' Pos:4 Stuct:0x1 ( FILE )
Adding:[1:diagnosis(10)] 'benign' Pos:5 Stuct:0x1 ( FILE )
Adding:[1:diagnosis(10)] 'colon' Pos:8 Stuct:0x1 ( FILE )
Adding:[1:diagnosis(10)] 'cancer' Pos:9 Stuct:0x1 ( FILE )
So you can see that they are all part of the same metaname "diagnosis", but
you can also see how the word positions are bumped (twice -- once for
ending tag, and again for start tag) to prevent matching the phrase "benign
colon" (thank goodness!).
>Think SWISH-E would find this document contains Prostate cancer>
> ./swish-e -w 'diagnosis=(prostate cancer)' -H0
1000 1.xml "1.xml" 116
Yes, it will find it. Is that what you want?
If not, then you would need to modify the docs a bit.
Swish-e has a magic bullet called -S prog. That means you can index any
document corpus you like since you can parse and reformat docs on-the-fly
while indexing to make them fit into how you want to search them.
Received on Tue Aug 20 16:34:32 2002