Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of shailendra.sharma@gmail.com
 designates 64.233.162.236 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=Gw4j2CFNenVh4O9o5tG9cuSN0y+kaHR39W/0AGd4mhDiweUJRDFNylIoL3780jU8POciQfXxVpgcWW05f4iZnHZ9SarMu5L2j6gb5OgkHjFhozjWoTXXeCmlptnwjYgZxjxS7/3vWg5ZDccNG++nhjGgKDt1iVkRB2r/vkXtQn4=
Message-ID: <d27fa17e0708031135i5692e3c6laf4328a09992dd80@mail.gmail.com>
Date: Sat, 4 Aug 2007 00:05:58 +0530
From: "Shailendra Sharma" <shailendra.sharma@gmail.com>
To: java-user@lucene.apache.org
Subject: Re: Can I do boosting based on term postions?
In-Reply-To: <200708031649.07438.paul.elschot@xs4all.nl>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_105989_7616100.1186166158743"
References: <839ba01c0707312113p278da04bg636e2bfa853e6ff4@mail.gmail.com>
	 <200708020922.55296.paul.elschot@xs4all.nl>
	 <839ba01c0708022038r7bf26d50nf4756773b76a9bdd@mail.gmail.com>
	 <200708031649.07438.paul.elschot@xs4all.nl>

------=_Part_105989_7616100.1186166158743
Content-Type: text/plain; charset=ISO-2022-JP
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Paul,

If I understand Cedric right, he wants to have different boosting depending
on search term positions in the document. By using SpanFirstQuery he will
only be able to consider in terms till particular position; but he won't be
able to do something like following:
  a) Give 100% boosting to matching in first 100 words.
  b) Give 80% boosting to matching in next 100 words.
  c) Give 60% boosting to matching in next 100 words.

Though it can be done by writing DisjunctionMaxQuery having multiple
SpanFirstQuery with different boosting - but I see it as a workaround only
and not the direct and efficient solution.

Cedric,

I am sending you the implementation of SpanTermQuery to your gmail
account (lucene
mailing list is bouncing email with attachment). I have named the class as
VSpanTermQuery (I have followed the same package hierarchy as lucene). You
also need to extend VSimilarity class - which would require implementation
of method scoreSpan(..).

Let me know how it went. Though I did a testing for it, but before
submitting to contrib, I need to do extensive testing.

Thanks,
Shailendra

On 8/3/07, Paul Elschot <paul.elschot@xs4all.nl> wrote:
>
> Cedric,
>
> You can choose the end limit for SpanFirstQuery yourself.
>
> Regards,
> Paul Elschot
>
>
> On Friday 03 August 2007 05:38, Cedric Ho wrote:
> > Hi Paul,
> >
> > Isn't SpanFirstQuery only match those with position less than a
> > certain end position?
> >
> > I am rather looking for a query that would score a document higher for
> > terms appear near the start but not totally discard those with terms
> > appear near the end.
> >
> > Regards,
> > Cedric
> >
> > On 8/2/07, Paul Elschot <paul.elschot@xs4all.nl> wrote:
> > > Cedric,
> > >
> > > SpanFirstQuery could be a solution without payloads.
> > > You may want to give it your own Similarity.sloppyFreq() .
> > >
> > > Regards,
> > > Paul Elschot
> > >
> > > On Thursday 02 August 2007 04:07, Cedric Ho wrote:
> > > > Thanks for the quick response =)
> > > >
> > > > On 8/1/07, Shailendra Sharma <shailendra.sharma@gmail.com> wrote:
> > > > > Yes, it is easily doable through "Payload" facility. During
> indexing
> > > process
> > > > > (mainly tokenization), you need to push this extra information in
> each
> > > > > token. And then you can use BoostingTermQuery for using Payload
> value
> to
> > > > > include Payload in the score. You also need to implement
> Similarity
> for
> > > this
> > > > > (mainly scorePayload method).
> > > >
> > > > If I store, say a custom boost factor as Payload, does it means that
> I
> > > > will store one more byte per term per document in the index file? So
> > > > the index file would be much larger?
> > > >
> > > > >
> > > > > Other way can be to extend SpanTermQuery, this already calculates
> the
> > > > > position of match. You just need to do something to use this
> position
> > > value
> > > > > in the score calculation.
> > > >
> > > > I see that SpanTermQuery takes a TermPositions from the indexReader
> > > > and I can get the term position from there. However I am not sure
> how
> > > > to incorporate it into the score calculation. Would you mind give a
> > > > little more detail on this?
> > > >
> > > > >
> > > > > One possible advantage of SpanTermQuery approach is that you can
> play
> > > > > around, without re-creating indices everytime.
> > > > >
> > > > > Thanks,
> > > > > Shailendra Sharma,
> > > > > CTO, Ver se' Innovation Pvt. Ltd.
> > > > > Bangalore, India
> > > > >
> > > > > On 8/1/07, Cedric Ho <cedric.ho@gmail.com> wrote:
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I was wondering if it is possible to do boosting by search
> terms'
> > > > > > position in the document.
> > > > > >
> > > > > > for example:
> > > > > > search terms appear in the first 100 words, or first 10% words,
> or
> in
> > > > > > first two paragraphs would be given higher score.
> > > > > >
> > > > > > Is it achievable through using the new Payload function in
> lucene
> 2.2?
> > > > > > Or are there any easier ways to achieve these ?
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > > Cedric
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > > > For additional commands, e-mail:
> java-user-help@lucene.apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> > > > Thanks,
> > > > Cedric
> > > >
> > > >
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > > >
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
> >
> > --
> > 愛@上.Keyboard
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

------=_Part_105989_7616100.1186166158743--