Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: Steven A Rowe <sarowe@syr.edu>
To: "java-user@lucene.apache.org" <java-user@lucene.apache.org>
Date: Tue, 23 Mar 2010 18:05:53 -0400
Subject: RE: Lucene query with long strings
Thread-Topic: Lucene query with long strings
Thread-Index: AcrKzQoiRj1s/7P5TPW3ZsJ2Mq8oBwABvJJw
Message-ID: 
 <2D127F11DC79714E9B6A43AC9458147F36662882@suex07-mbx-03.ad.syr.edu>
References: <369772.63337.qm@web50706.mail.re2.yahoo.com>
In-Reply-To: <369772.63337.qm@web50706.mail.re2.yahoo.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi Aaron,

Your "false positives" comments point to a mismatch between what you're cur=
rently asking Lucene for (any document matching any one of the terms in the=
 query) and what you want (only fully "correct" matches).

You need to identify the terms of the query that MUST match and tell Lucene=
 about it ("+" syntax is understood by QueryParser to mean a required term)=
.

If your queries come from sources that don't reliably match the indexes val=
ues, you may need to use synonyms to map between e.g. "California" and "CA"=
, and then require that at least one of the synonyms matches (e.g. "+(Calif=
ornia CA)").

Steve

On 03/23/2010 at 5:08 PM, Aaron Schon wrote:
> hi all, I have been playing with=A0Lucene for a while now, but stuck on a
> perplexing issue.
>=20
> I have an index, with a field "Affiliation",=A0some example values are:
>=20
> - "Stanford University School of Medicine, Palo Alto, CA USA", -
> "Institute of Neurobiology, School of Medicine, Stanford University,
> Palo Alto, CA", - "School of Medicine, Harvard University, Boston MA", -
> "Brigham & Women's, Harvard University School of Medicine, Boston, MA" -
> "Harvard University, Cambridge MA"
>=20
> and so on... (the bottom-line being the affiliations are written in
> multiple ways with no apparent consistency)
>=20
> I query the index on=A0 the affiliation field using say "School of
> Medicine, Stanford University, Palo Alto, CA" (with QueryParser)=A0to
> find all Stanford related documents, I get a lot of false +ves,
> presumably because of the presence of School of Medicine etc. etc.
> (note: I cannot use Phrase query because of variability in the way
> affiliation is constructed)
>=20
> I have tried the following:
>=20
> 1. Use a SpanNearQuery by splitting the search phrase with a whitespace
> (here I get no results!)
> 2. Tried boosting (using ^) by splitting with the comma and boosting
> the last parts such as "Palo Alto CA" with a much higher boost than the
> initial phrases. Here I still get lots of false +ves.
>=20
> Any suggestions on how to approach this? Is SpanNear the way to go? Any
> other ideas on why I get 0 results?
>=20
> Thanks in advance for helping a newbie.
>=20
> AS


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org