lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <>
Subject [jira] [Commented] (LUCENE-2956) Support updateDocument() with DWPTs
Date Tue, 12 Apr 2011 06:54:05 GMT


Simon Willnauer commented on LUCENE-2956:

bq. I think I have an idea, however can you explain the ticketQueue?

Sure, since with DWPT the flush process is concurrent and several DWPT could flush at the
same time we must maintain the order of the flushes before we can apply the flushed segment
and the frozen global
deletes it is buffering. The reason for this is that the global deletes mark a certain point
in time where we took a DWPT out of rotation and freeze the global deletes.

Example: A DWPT 'A' starts flushing and freezes the global deletes, then DWPT 'B' starts flushing
and freezes all deletes occurred since 'A' has started. if 'B' finishes before 'A' we need
to wait until 'A' is done otherwise the deletes frozen by 'B' are not applied to 'A' and we
might miss to deletes documents in 'A'.

The Ticket queue simply ensures that we push the frozen deletes and the new created segment
in the same order as the corresponding DWPT have started flushing. If a DWPT finishes flushing
we update its Ticket and then check the head of the queue if we can remove / publish the ticket.
if so we continue publishing until the head of the queue can not be published yet or the queue
is empty.

> Support updateDocument() with DWPTs
> -----------------------------------
>                 Key: LUCENE-2956
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: Realtime Branch
>            Reporter: Michael Busch
>            Assignee: Simon Willnauer
>            Priority: Minor
>             Fix For: Realtime Branch
>         Attachments: LUCENE-2956.patch
> With separate DocumentsWriterPerThreads (DWPT) it can currently happen that the delete
part of an updateDocument() is flushed and committed separately from the corresponding new
> We need to make sure that updateDocument() is always an atomic operation from a IW.commit()
and IW.getReader() perspective.  See LUCENE-2324 for more details.

This message is automatically generated by JIRA.
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message