Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of Lawrence@theladders.com
 designates 67.151.144.115 as permitted sender)
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----_=_NextPart_001_01C6960B.008AF056"
Subject: RE: Lucene and SIPs
Date: Thu, 22 Jun 2006 10:49:02 -0400
Message-ID: <15FC71D155A0F8429C3636B72C2ED52E19BF39@hermes.laddersoffice.com>
Thread-Topic: Lucene and SIPs
thread-index: AcaVkL2nmor4fsAaRiKQdhu/TCSvswAeJFVA
From: "Larry Ogrodnek" <Lawrence@theladders.com>
To: "Nader Akhnoukh" <iamnader@gmail.com>
Cc: <java-user@lucene.apache.org>

------_=_NextPart_001_01C6960B.008AF056
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

I didn't make too much progress, and kind of ended up dropping it.

=20

One thing that I played with was creating multiple phrase indexes, one
each for 2, 3, 4, and 5 words.  I wrote a tokenizer that would batch up
the words, so, for the input string:

=20

The quick brown fox jumps over the slow lazy dog.

=20

The tokenizer for 3 words would return:

=20

The quick brown

Quick brown fox

Brown fox jumps

Fox jumps over

...

=20

This seemed like a reasonably start... the problem is resolving the
overlap for display, and figuring out which words are the most
important, e.g. if the above sentence itself was pretty rare, and you're
looking at the phrase-index-3, each one of its sub-phrases would end up
being significant.... Which one do you show?  Or do you combine them
into a longer phrase?  If so, where do you stop?

=20

It seemed like an easy first-approach to try out, but I'm not sure it's
even in the right direction...

=20

=20

=20

=20

________________________________

From: Nader Akhnoukh [mailto:iamnader@gmail.com]=20
Sent: Wednesday, June 21, 2006 8:14 PM
To: Larry Ogrodnek
Subject: Lucene and SIPs

=20

Hi Lawrence, I saw a posting to the Lucene group you made in February
concerning using Lucene to find SIPs.

Did you make any progress with this?  I'm able to find significant
single terms, but am stumped by phrases.=20

Thanks,
Nader


------_=_NextPart_001_01C6960B.008AF056--