lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: shared doc stores and IW.addIndexes
Date Mon, 23 May 2011 18:11:58 GMT
>
> It would be nice if addIndexes "fixed" this for you... but I think not
> necessary?


I agree. If the code can handle old segments w/ shared doc stores, there's
no reason to introduce another complexity to addIndexes. It should be an
optimized method. The shared doc stores will go away when segments are
merged.

Do we plan to support this forever, or should we mark this feature and code
handling for removal when 4.0 (or likely 5.0, since 3.0 still write shared
doc stores) is out?

Shai

On Mon, May 23, 2011 at 6:58 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> We no longer write shared doc stores, in both 3.x and 4.0, but we are
> able to read them (from an older index).
>
> Merging works fine... merging will copy out the slice of the shared
> doc store that it needs.
>
> It would be nice if addIndexes "fixed" this for you... but I think not
> necessary?
>
> Mike
>
> http://blog.mikemccandless.com
>
> On Mon, May 23, 2011 at 8:39 AM, Shai Erera <serera@gmail.com> wrote:
> > Hi
> >
> > If I remember correctly, we no longer share doc stores (starting 3.1.0).
> > While working on LUCENE-3126 I was thinking that perhaps addIndexes does
> not
> > work well, or at least can be improved in that area.
> >
> > Today, if you add an index (e.g. created with 2.9/3.0) and some segments
> > share doc stores, they will still share the doc stores after copy.
> However,
> > what if you use a merge policy, e.g. TieredMP, which assumes it can merge
> > segments out-of-order? Will it work w/ shared doc-stores? How does the
> rest
> > of the code work w/ segments that share doc stores?
> >
> > I assume that since we support index-format backwards, then the code
> works
> > well. So maybe this isn't a bug. However, shouldn't we untie the doc
> stores,
> > even if it means copying the DS to each segment, in IW.addIndexes?
> >
> > Shai
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Mime
View raw message