Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Mon, 5 Nov 2012 22:22:12 +0000 (UTC)
From: "Adrien Grand (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <1813417739.70934.1352154132905.JavaMail.jiratomcat@arcas>
In-Reply-To: <115748518.97638.1348068548196.JavaMail.jiratomcat@arcas>
Subject: [jira] [Commented] (SOLR-3855) DocValues support
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/SOLR-3855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490984#comment-13490984 ] 

Adrien Grand commented on SOLR-3855:
------------------------------------

bq. We could combine these? e.g. a docValueType of "none" or something?

Something I like about having two different parameters is that it gives the ability to specify a default DocValues.Type (used when docValues is true and docValueType is unset). For example, with some FieldTypes only one DocValues.Type makes sense (UUIDField -> FIXED_STRAIGHT) and it would make sense to make FIXED_INTS_32 the default docValueType for TrieIntField.

Additionally, maybe that not having to learn about all DocValues types before using them would make their adoption smoother?

bq. So this way there is no trap (slow-io, or huge ram) on anyone.

I like this argument (forcing the trade-off to be explicit)! But this makes me want to merge DocValues activation with the method instead of the type. For example we could say docValues=no|disk|memory (optional, defaults to "no", "disk" to enable direct doc values, "memory" otherwise) docValueType=${type} (optional, default value depends on the FieldType, only taken into account when "docValues" is set and is not "no").

I don't like the fact that I needed to add createDocValues method in addition to the createField method, but only poly fields can return several fields. We should probably fix the API (maybe something like normal fields must have at most one indexed field but should be able to produce several stored fields?) but I'd rather do it in a different issue.

bq. for this initial patch, we could just make it so only "memory" actually works and you get a UOE if you ask for direct. I feel like this is a good step to remove uninversion times, etc and we could separately add the disk option...

Adding support for direct doc values should be easy: DocValues value sources already support direct DocValues, we just need to fix oal.search.FieldComparator implementations to have an option to use direct DocValues (they currently force in-memory DocValues).
                
> DocValues support
> -----------------
>
>                 Key: SOLR-3855
>                 URL: https://issues.apache.org/jira/browse/SOLR-3855
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Adrien Grand
>            Assignee: Adrien Grand
>            Priority: Minor
>             Fix For: 4.1, 5.0
>
>         Attachments: SOLR-3855.patch
>
>
> It would be nice if Solr supported DocValues:
>  - for ID fields (fewer disk seeks when running distributed search),
>  - for sorting/faceting/function queries (faster warmup time than fieldcache),
>  - better on-disk and in-memory efficiency (you can use packed impls).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org