lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-3065) NumericField should be stored in binary format in index (matching Solr's format)
Date Wed, 04 May 2011 19:06:03 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Uwe Schindler updated LUCENE-3065:
----------------------------------

    Attachment: LUCENE-3065-solr-only.patch

Here a first step in cutover of Solr to NumericField. Most tests work, except:
- TestDistributedSearch, fails with a strange date problem - I have no idea what goes wrong
- TestMoreLikeThis: fails because the returned documents are different than expected. The
reason for this is simple: As TrieField's underlying Lucene fields now are NumericField, stringValue()
returns something (in contrast, solr's old fields returned null because they were binary).
This confuses maybe MoreLikeThis (needs maybe fixed in Lucene, I havent looked into the code).
Maybe we should simply exclude those fields or fix the test (I prefer latter one, because
the numerics should also taken into account).

The following changes had to be made:
- Cut over all places in Solr where Field instead of abstract Fieldable is used to Fieldable.
This affects some leftover parts in various components (calling Document.getField instead
of Document.getFieldable), but mainly SchemaField/FieldType: createField() now returns Fieldable
- TrieDateField code duplication was removed, all methods delegate to a wrapped TrieField.
There was also an inconsitency between TrieField and TrieDateField's toExternal(). This was
fixed to work correct (the date format was wrong, now it uses dateField.toExternal())

If somebody could help with the rest of the solr stuff and maybe test test test! Yonik? Ryan?
There may be some itches not covered by tests.

Thanks for help from Solr specialists (I am definitely not one, I am more afraid of the code
than I can help)!!!

> NumericField should be stored in binary format in index (matching Solr's format)
> --------------------------------------------------------------------------------
>
>                 Key: LUCENE-3065
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3065
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>            Reporter: Michael McCandless
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.2, 4.0
>
>         Attachments: LUCENE-3065-solr-only.patch, LUCENE-3065.patch, LUCENE-3065.patch,
LUCENE-3065.patch, LUCENE-3065.patch, LUCENE-3065.patch, LUCENE-3065.patch, LUCENE-3065.patch
>
>
> (Spinoff of LUCENE-3001)
> Today when writing stored fields we don't record that the field was a NumericField, and
so at IndexReader time you get back an "ordinary" Field and your number has turned into a
string.  See https://issues.apache.org/jira/browse/LUCENE-1701?focusedCommentId=12721972&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12721972
> We have spare bits already in stored fields, so, we should use one to record that the
field is numeric, and then encode the numeric field in Solr's more-compact binary format.
> A nice side-effect is we fix the long standing issue that you don't get a NumericField
back when loading your document.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message