Thanks Tim.
I have thought about this for the author field (and like you suggest) it would probably work.
I was actually going to experiment with this later today.
But, I have another field that has a bit more nesting (and it contains authors)
For example, within a given document, I have the following:
References [ one or more]
Authors [one or more]
First Name
Last Name
So, I would need to search for a specific author (matching first name and last name) within
a specific reference for a document. With this double level of nesting, I don't think the
multivalued field approach would work (please correct me if I'm wrong). That's why I decided
to use span queries. My index has more than 100 fields, but I only have 2 or 3 fields that
require this structure search capability. There are also many documents (100M) so I didn't
really want to get into a parent-child type approach.
Plus, there are also many other fields (both within an author) and within an individual reference
that need to be scoped. For example, there is a 'source tittle' at the reference level and
an 'article title' at the reference level. I would need to search within a given reference
where the 'source title' contains some words, where the 'article title' contains some words,
and within this reference where a specific author contains 'john' for the first name and 'smith'
in the last name.
I guess I'm curious if what I was doing with the SpanQuery should have worked, whether I misunderstood
something, or if this is a bug.
Darin.
________________________________
From: "Allison, Timothy B." <tallison@mitre.org>
To: "java-user@lucene.apache.org" <java-user@lucene.apache.org>; Darin McBeath <ddmcbeath@yahoo.com>
Sent: Friday, June 6, 2014 10:12 AM
Subject: RE: SpanQuery not working as expected
Hi Darin,
Have you thought about using multivalued fields? If you set the positionIncrementGap to
something kind of big (well > 1, say :) ), and you know that your data is always authorfirst,
authorlast, you could just search for "darin fulford".
The positionincrementgap will prevent matching on doc2 below.
Doc1
Authorsfield:
Darin fulford
Doc2
Authorsfield:
Matilda darin
Fulford alexandria
Don't get me wrong, I love the capabilities of SpanQuery, but will this simple solution meet
your needs?
-----Original Message-----
From: Darin McBeath [mailto:ddmcbeath@yahoo.com.INVALID]
Sent: Thursday, June 05, 2014 7:17 PM
To: java-user@lucene.apache.org
Subject: SpanQuery not working as expected
I read through the http://searchhub.org/2009/07/18/the-spanquery/ which provided a good
overview for how one can construct fairly complex span queries. I was particularly interested
in the ability to construct nested span queries. I'm trying to apply this concept to search
a field that contains some structure (as below). I have a couple of other fields that will
have a bit more nesting, but this should give the general idea.
authors
author [one or more]
first name
last name
Prior to indexing the content with Lucene, I added some 'markers' around the various bits
I might want to search. For example 'bauthor' implies beginning author, 'eauthor' implies
ending author, and 'sauthor' implies a separator between individual authors (that would be
used as part of the exclude clause in a not span query). I do similar things for 'first
name' and 'last name'.
My constructed query (as interpreted by Lucene) is included below. This was extracted from
the 'parsed string' returned from the query when I set debug=true. Within a given 'authscope'
field, I'm trying to find a situation where the author first name is 'darin' and the last
name is 'fulford' within a given 'author'.
spanNot(
spanNear(
[authscope:bauthor,
spanNear(
[spanNot(
spanNear(
[authscope:bfname,
authscope:darin,
authscope:efname],
2147483647, true),
authscope:sfname, 0, 0),
spanNot(
spanNear(
[authscope:blname,
authscope:fulford,
authscope:elname],
2147483647, true),
authscope:slname, 0, 0)],
2147483647, false),
authscope:eauthor],
2147483647, true),
authscope:sauthor, 0, 0)",
I have loaded the following 2 documents into my index.
[
{"id":"1", "authscope":" bauthors bauthor blname mcbeath elname slname bfname
darin efname sfname eauthor sauthor bauthor blname fulford elname slname bfname
darby efname sfname eauthor sauthor bauthor blname mcbeath elname slname bfname
darby efname sfname eauthor sauthor eauthors sauthors "},
{"id":"2", "authscope":" bauthors bauthor blname mcbeath elname slname bfname
darin efname sfname eauthor sauthor bauthor blname fulford elname slname bfname
darin efname sfname eauthor sauthor eauthors sauthors "}
]
What I can't figure out is why the above query would match on both documents. It should
only match the document with id:2.
Any insights would be appreciated. I'm using Lucene 4.7.2.
Darin.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
|