lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <>
Subject [jira] [Commented] (LUCENE-2025) Ability to turn off the store for an index
Date Fri, 31 Aug 2012 14:41:08 GMT


Robert Muir commented on LUCENE-2025:

One simple way to do this today is to just use a codec that has a NoStoredFieldsImpl,
Throws exception in its writer impl if you ask it to actually write any stored fields 
(e.g. startDocument\(n\) is called where n > 0), and does nothing in its reader impl.

I think for the typical case its fairly uncommon, i looked into seeing if we could
optimize this case for Lucene40's impl, but it introduces a lot of scary situations
for things like bulk merge.

So for now I really think this is a simple safe way at the moment, if someone wants to 
turn it off they just set this as their codec on indexwriter.

> Ability to turn off the store for an index
> ------------------------------------------
>                 Key: LUCENE-2025
>                 URL:
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: core/index
>            Reporter: Michael Busch
>            Assignee: Michael Busch
>            Priority: Minor
>              Labels: gsoc2011, gsoc2012, lucene-gsoc-11, lucene-gsoc-12, mentor
>             Fix For: 4.1
> It would be really good in combination with parallel indexing if the
> Lucene store could be turned off entirely for an index. 
> The reason is that part of the store is the FieldIndex (.fdx file),
> which contains an 8 bytes pointer for each document in a segment, even
> if a document does not contain any stored fields.
> With parallel indexing we will want to rewrite certain parallel
> indexes to update them, and if such an update affects only a small
> number of documents it will be a waste if you have to write the .fdx
> file every time.
> So in the case where you only want to update a data structure in the
> inverted index it makes sense to separate your index into multiple
> parallel indexes, where the ones you want to update don't contain any
> stored fields.
> It'd be also great to not only allow turning off the store but to make
> it customizable, similarly to what flexible indexing wants to achieve
> regarding the inverted index.
> As a start I'd be happy with the ability to simply turn off the store and to
> add more flexibility later.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message