lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-3216) Store DocValues per segment instead of per field
Date Thu, 30 Jun 2011 09:34:28 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Simon Willnauer updated LUCENE-3216:
------------------------------------

    Attachment: LUCENE-3216.patch

we are getting closer to the overall target here. This patch enables each codec to decided
to use CFS for DocValues or write individual files. 

To configure this and more stuff per codec I introduced a CodecConfig (just like IWC) that
holds configuration for core codecs and is passed to each codec on creation. I added testcases
for the Config and for nested CFS in the case IW or SegmentMerger decides to use CFS too so
DocValues still can safely open the CFS.

For test coverage I added a static newCodecConfig() to LuceneTestCase that randomly configures
a codec per file to use CFS or individual files for DocValues and other stuff I figured make
sense in the CodecConfig.

All tests pass and there is no nocommit left I think its close. Review is appreciated

> Store DocValues per segment instead of per field
> ------------------------------------------------
>
>                 Key: LUCENE-3216
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3216
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/index
>    Affects Versions: 4.0
>            Reporter: Simon Willnauer
>            Assignee: Simon Willnauer
>             Fix For: 4.0
>
>         Attachments: LUCENE-3216.patch, LUCENE-3216.patch, LUCENE-3216.patch, LUCENE-3216.patch,
LUCENE-3216.patch, LUCENE-3216_floats.patch
>
>
> currently we are storing docvalues per field which results in at least one file per field
that uses docvalues (or at most two per field per segment depending on the impl.). Yet, we
should try to by default pack docvalues into a single file if possible. To enable this we
need to hold all docvalues in memory during indexing and write them to disk once we flush
a segment. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message