lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Rutherglen (JIRA)" <>
Subject [jira] Updated: (LUCENE-2047) IndexWriter should immediately resolve deleted docs to docID in near-real-time mode
Date Mon, 23 Nov 2009 02:53:39 GMT


Jason Rutherglen updated LUCENE-2047:

    Attachment: LUCENE-2047.patch

In the updateDocument and deleteDocument methods, deletes are
buffered per segment reader synchronized on writer. Immediately
after, outside the sync block, they're deleted from the existing
SRs. If a new SR is added, it's because of a flush (which has
it's own buffered deletes), or from a merge which is
synchronized on writer. In other words, we won't lose deletes,
they'll always be applied on a flush, and the resolution of
deletes to doc ids happens un-synchronized on writer.

Update document adds the term to the queued deletes, then
resolves and adds the doc ids to an Integer list (for now). This
is where we may want to use an growable int[] or int set.

Flush applies queued update doc deleted doc ids to the SRs.

commitMerge merges queued deletes from the incoming SRs. Doc ids
are mapped to the merged reader.

> IndexWriter should immediately resolve deleted docs to docID in near-real-time mode
> -----------------------------------------------------------------------------------
>                 Key: LUCENE-2047
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 3.1
>         Attachments: LUCENE-2047.patch, LUCENE-2047.patch, LUCENE-2047.patch, LUCENE-2047.patch
> Spinoff from LUCENE-1526.
> When deleteDocuments(Term) is called, we currently always buffer the
> Term and only later, when it's time to flush deletes, resolve to
> docIDs.  This is necessary because we don't in general hold
> SegmentReaders open.
> But, when IndexWriter is in NRT mode, we pool the readers, and so
> deleting in the foreground is possible.
> It's also beneficial, in that in can reduce the turnaround time when
> reopening a new NRT reader by taking this resolution off the reopen
> path.  And if multiple threads are used to do the deletion, then we
> gain concurrency, vs reopen which is not concurrent when flushing the
> deletes.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message