Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of simon.willnauer@googlemail.com
 designates 72.14.220.154 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=mime-version:reply-to:in-reply-to:references:date:message-id
         :subject:from:to:content-type;
        b=EVGNL45PKr9Q2d/JZw0p5G/IJm7wLnayB/S04f0pHVpeUmVmPboftCYSGI1rFA7BFu
         dbDXBU6Tb7SdsoOUUCG8ZD7oCx4O/Q2/kAb6fT7MyPjCSUOm4VSNnG47I3QRGZJOnSjL
         mDkOkflZ797BpEELVZdrsuSvkccbSk/WeIWS4=
MIME-Version: 1.0
Reply-To: simon.willnauer@gmail.com
In-Reply-To: <8d3740db0911080523sd3755c6o55cf65abb109cf47@mail.gmail.com>
References: <8d3740db0911080523sd3755c6o55cf65abb109cf47@mail.gmail.com>
Date: Sun, 8 Nov 2009 14:46:32 +0100
Message-ID: <f18c9dde0911080546p17a4356cvc3bba4c2bf3b9973@mail.gmail.com>
Subject: Re: synonym payload boosting
From: Simon Willnauer <simon.willnauer@googlemail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=001485f27c200531970477dc4e69

--001485f27c200531970477dc4e69
Content-Type: text/plain; charset=UTF-8

You might get an answer on the solr list. This is the lucene users list.

Simon

On Nov 8, 2009 2:24 PM, "David Ginzburg" <davidginzburg@gmail.com> wrote:

Hi,
I have a field and a wighted synonym map.
I have indexed the synonyms with the weight as payload.
my code snippet from my filter

*public Token next(final Token reusableToken) throws IOException *
*        . *
*        . *
*        .*
      * Payload boostPayload;*
*
*
*        for (Synonym synonym : syns) {*
*            *
*            Token newTok = new Token(nToken.startOffset(),
nToken.endOffset(), "SYNONYM");*
*            newTok.setTermBuffer(synonym.getToken().toCharArray(), 0,
synonym.getToken().length());*
*            // set the position increment to zero*
*            // this tells lucene the synonym is*
*            // in the exact same location as the originating word*
*            newTok.setPositionIncrement(0);*
*            boostPayload = new
Payload(PayloadHelper.encodeFloat(synonym.getWieght()));*
*            newTok.setPayload(boostPayload);*
*
*
I have put it in the index time analyzer : this is my field definition:

*
<fieldType name="PersonName" class="solr.TextField"
positionIncrementGap="100" >
     <analyzer type="index">
       <tokenizer class="solr.WhitespaceTokenizerFactory"/>
       <filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
       <filter class="solr.LowerCaseFilterFactory"/>
       <filter class="com.digitaltrowel.solr.DTSynonymFactory"
FreskoFunction="names_with_scoresPipe23Columns.txt" ignoreCase="true"
expand="false"/>

       <!--<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/>-->
       <!--<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>-->
     </analyzer>
     <analyzer type="query">
       <tokenizer class="solr.WhitespaceTokenizerFactory"/>
       <filter class="solr.LowerCaseFilterFactory"/>
       <!--<filter class="com.digitaltrowel.solr.DTSynonymFactory"
synonyms="synonyms.txt" ignoreCase="true" expand="false"/>-->
       <filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
       <!--<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/>-->
       <!--<filter class="solr.RemoveDuplicatesTokenFilterFactory"/    >-->
     </analyzer>
   </fieldType>


my similarity class is
public class BoostingSymilarity extends DefaultSimilarity {


   public BoostingSymilarity(){
       super();

 }
   @Override
   public  float scorePayload(String field, byte [] payload, int offset,
int length)
{
 double weight = PayloadHelper.decodeFloat(payload, 0);
return (float)weight;
 }

@Override public float coord(int overlap, int maxoverlap)
 {
return 1.0f;
}

@Override public float idf(int docFreq, int numDocs)
{
 return 1.0f;
}

@Override public float lengthNorm(String fieldName, int numTerms)
 {
return 1.0f;
}

@Override public float tf(float freq)
{
 return 1.0f;
}
}

My problem is that scorePayload method does not get called at search time
like the other methods in  my similarity class.
I tested and verified it with break points.
What am I doing wrong?
I used solr 1.3 and thinking of the payload boos support in solr 1.4.


*

--001485f27c200531970477dc4e69--