lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-2025) Ability to turn off the store for an index
Date Fri, 31 Aug 2012 14:41:08 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446003#comment-13446003
] 

Robert Muir commented on LUCENE-2025:
-------------------------------------

One simple way to do this today is to just use a codec that has a NoStoredFieldsImpl,
Throws exception in its writer impl if you ask it to actually write any stored fields 
(e.g. startDocument\(n\) is called where n > 0), and does nothing in its reader impl.

I think for the typical case its fairly uncommon, i looked into seeing if we could
optimize this case for Lucene40's impl, but it introduces a lot of scary situations
for things like bulk merge.

So for now I really think this is a simple safe way at the moment, if someone wants to 
turn it off they just set this as their codec on indexwriter.

                
> Ability to turn off the store for an index
> ------------------------------------------
>
>                 Key: LUCENE-2025
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2025
>             Project: Lucene - Core
>          Issue Type: New Feature
>          Components: core/index
>            Reporter: Michael Busch
>            Assignee: Michael Busch
>            Priority: Minor
>              Labels: gsoc2011, gsoc2012, lucene-gsoc-11, lucene-gsoc-12, mentor
>             Fix For: 4.1
>
>
> It would be really good in combination with parallel indexing if the
> Lucene store could be turned off entirely for an index. 
> The reason is that part of the store is the FieldIndex (.fdx file),
> which contains an 8 bytes pointer for each document in a segment, even
> if a document does not contain any stored fields.
> With parallel indexing we will want to rewrite certain parallel
> indexes to update them, and if such an update affects only a small
> number of documents it will be a waste if you have to write the .fdx
> file every time.
> So in the case where you only want to update a data structure in the
> inverted index it makes sense to separate your index into multiple
> parallel indexes, where the ones you want to update don't contain any
> stored fields.
> It'd be also great to not only allow turning off the store but to make
> it customizable, similarly to what flexible indexing wants to achieve
> regarding the inverted index.
> As a start I'd be happy with the ability to simply turn off the store and to
> add more flexibility later.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message