lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emir Arnautović <emir.arnauto...@sematext.com>
Subject Re: get all tokens from TokenStream in my custom filter
Date Thu, 16 Nov 2017 09:26:45 GMT
Hi,
Maybe you are not pasting entire filter code, but you need to set attributes and return true.
Take a look at come other token filter how it is done.
Here is how it is done in UpperCaseFilter:
  public final boolean incrementToken() throws IOException {
    if (input.incrementToken()) {
      charUtils.toUpperCase(termAtt.buffer(), 0, termAtt.length());
      return true;
    } else
      return false;
  }

termAtt buffer is updated with new value. You need to adjust position, length and term attributes.

HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 16 Nov 2017, at 07:56, Modassar Ather <modather1981@gmail.com> wrote:
> 
> Hi,
> 
> Are you getting the string but further tokens are not generated? If this is
> the case I think no need to end or close the stream in filter.
> Hope this helps.
> 
> Best,
> Modassar
> 
> On Thu, Nov 16, 2017 at 1:20 AM, kumar gaurav <kg2989@gmail.com> wrote:
> 
>> Hi
>> 
>> I need to get full field value from TokenStream in my custom filter class .
>> 
>> I am using this
>> 
>> stream.reset();
>> while (tStream.incrementToken()) {
>>    term += " "+charTermAttr.toString();
>> }
>> stream.end();
>> stream.close();
>> 
>> this is ending streaming . no token is producing if i am using this .
>> 
>> I want to get full string without hampering token creation .
>> 
>> Eric ! Are you there ? :)  Anyone Please help  ?
>> 


Mime
View raw message