lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: Why Numeric (IntField) tokenized true by Default?
Date Mon, 03 Jun 2013 10:41:19 GMT
This is because, under the hood, a single number is indexed into
several "hierarchical" tokens, to make NumericRangeFilter/Query very

When this field is sorted, there is careful logic to use only the
original number for sorting purposes

So, it's true that in general you should not tokenize a field that you
will sort on, but numeric fields are an exception to this rule, and
sorting will work correctly.

Mike McCandless

On Mon, Jun 3, 2013 at 4:00 AM, Arjun Dhar <> wrote:
> This post was updated on Jun 03, 2013; 3:48am.
> If One refers to the JavaDoc for Sort it states that INDEX and and the field
> should NOT be TOKENIZED.
> Its a common use case for Numbers to be Sorted.
> Am curious to find the default values of say IntField as :
> TYPE_NOT_STORED.setTokenized(true);
> Any reason why its tokenized by default?
> The constructors suggest you should play with STORE or NO STORE @ best. But
> I feel there is sufficient reason to have to define your own FieldType for
> sorting ?
> --
> View this message in context:
> Sent from the Lucene - Java Users mailing list archive at
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message