lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: addIndexes* blocks addDocuments calls
Date Tue, 14 Jul 2009 19:56:11 GMT
I agree, there's no real reason why addIndexes can run concurrently
with other things.  It's just software ;)

One challenge is the transactional guarantee that addIndexes provide,
ie, it's all or none.  If there's an exception while adding, then
nothing was added.

But, that was added before autoCommit=false.  So perhaps we could
relax that and expect the app to instead rely on the "global"
transactional semantics provided by autoCommit=false.

EG you could imagine an addIndexes* call getting started, completing a
few merges.  Then, concurrently, CMS picks & chooses some of those
added external segments to merge with some of the original segments.
Then addIndexes hits an exception.  What do we do?

If IndexWriter maintained "branches" of the segmentInfos we could
actually rollback all changes, ie, remove what was done by addIndexes
but retroactively preserve any segments created by other methods
(flushing, other addIndexes calls, etc.).

Mike

On Tue, Jul 14, 2009 at 3:08 PM, Jason
Rutherglen<jason.rutherglen@gmail.com> wrote:
> For replicating and general system performance, it would be good to offer a
> way to addIndexes* without blocking the addition of more docs. This seems
> doable somehow?

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message