lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-2655) Get deletes working in the realtime branch
Date Wed, 13 Oct 2010 17:14:30 GMT


Michael McCandless commented on LUCENE-2655:

Just to confirm: when a delete is done, we go and buffer that delete into each DWPT, right?
 (Mapped to the then-current docid-upto for that DWPT).

OK I see the problem.  It's because we now have a single pool for deletesFlushed, right? 
Ie, DWPT #2 will overwrite the term id:1 entry.

But, I think the switch to generations of pending deletes (LUCENE-2680) would fix this?  Maybe
we should go do that one first...

Ie, DWPT #1's flush would enter a new gen delete pool (maps term -> global docid-upto).
 Then DWPT #2's flush would also enter a new gen delete pool.  Hmm, but not quite... the generations
can't simply stack on top of one another.  I think there's a graph structure somehow?  Ie
every DWPT that's flushed must record the segments that existed (were already flushed) when
it was first created, because it's only those segments that should get the deleted term. 
Segments in the future obviously shouldn't get it.  And segments from parallel DWPTs (ie that
existed at the same time) should also not get it since they will separately track the deleted

BTW, I think this makes LUCENE-2679 all the more important (the ability to delete such that
delete will only apply to already-committed segments), since this'd mean we only store the
pending delete in a single map instead of map per DWPT.

> Get deletes working in the realtime branch
> ------------------------------------------
>                 Key: LUCENE-2655
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: Realtime Branch
>            Reporter: Jason Rutherglen
>             Fix For: Realtime Branch
>         Attachments: LUCENE-2655.patch
> Deletes don't work anymore, a patch here will fix this.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message