lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-3218) Make CFS appendable
Date Wed, 24 Aug 2011 08:28:29 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Simon Willnauer updated LUCENE-3218:
------------------------------------

    Attachment: LUCENE-3218.patch

new patch, I renamed IndexInputHandle to IndexInputSlicer and made the createSlicer method
public otherwise Directory impls outside of o.a.l.store can not delegate to it.

bq.I would also rename CFIndexInput to SliceIndexInput, it's private so does not matter, but
wozuld be nice to have.

done

bq. Otherwise I agree with committing to trunk. As far as I see, the format did not change
in trunk, so once we get this back into 3.x we are at the state pre-revert?

yes that's true.

I think is ready to commit, if nobody objects I am going to commit this later today.

> Make CFS appendable  
> ---------------------
>
>                 Key: LUCENE-3218
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3218
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/index
>    Affects Versions: 3.4, 4.0
>            Reporter: Simon Willnauer
>            Priority: Blocker
>             Fix For: 3.4, 4.0
>
>         Attachments: LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch,
LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218.patch, LUCENE-3218_3x.patch, LUCENE-3218_test_fix.patch,
LUCENE-3218_tests.patch
>
>
> Currently CFS is created once all files are written during a flush / merge. Once on disk
the files are copied into the CFS format which is basically a unnecessary for some of the
files. We can at any time write at least one file directly into the CFS which can save a reasonable
amount of IO. For instance stored fields could be written directly during indexing and during
a Codec Flush one of the written files can be appended directly. This optimization is a nice
sideeffect for lucene indexing itself but more important for DocValues and LUCENE-3216 we
could transparently pack per field files into a single file only for docvalues without changing
any code once LUCENE-3216 is resolved.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message