lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <erik.hatc...@gmail.com>
Subject Re: Lucene TypeAttribute not used during querying
Date Wed, 23 Sep 2015 13:15:53 GMT
Paul -

There are a couple of TokenFilter’s of note in Lucene that leverage the “type” attribute
that may help:

  - TypeAsPayloadTokenFilter: sets the payload of the terms to the type attribute value
  - TypeTokenFilter: includes or excludes tokens that have are in a designated set of types

And there’s also TokenTypeSinkFilter that can be used with a TeeSinkTokenFilter to tee tokens
with a particular type.

—
Erik Hatcher, Senior Solutions Architect
http://www.lucidworks.com




> On Sep 23, 2015, at 5:38 AM, Paul Bedaride <paul.bedaride@xilopix.com> wrote:
> 
> Ok so it is not possible to store other part of information in the index ? like part-of-speach
?
> 
> Thanks for the fast answer
> 
> Paul
> 
> On 23/09/2015 11:21, Uwe Schindler wrote:
>> Hi,
>> 
>> The type attribute is not stored in index. The main intention behind this attribute
is to use it inside the analysis chain. E.g. you have some tokenizer/stemmer/whatever that
sets the attribute. The last TokenFilter before indexing may then change the term accordingly
(e.g. adding the type as a payload, or append it to the term itsself) to get the information
into index - but this is mainly your task. The same applies for other language specific attributes
(like Japanese ones). The keyword attribute is another example, it is also not indexed, but
is solely used to control behavior of later TokenFilters (e.g. prevent stemming).
>> 
>> Uwe
>> 
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>> 
>> 
>>> -----Original Message-----
>>> From: Paul Bedaride [mailto:paul.bedaride@xilopix.com]
>>> Sent: Wednesday, September 23, 2015 11:16 AM
>>> To: general@lucene.apache.org
>>> Subject: Lucene TypeAttribute not used during querying
>>> 
>>> Hello,
>>> 
>>> I wonder why the TypeAttribute is not used for queries ?
>>> It seems that it is used only during analysis.
>>> Why it is not used in org.apache.lucene.index.Term ?
>>> 
>>> Paul Bédaride
> 


Mime
View raw message