Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <1572841369.1236213836333.JavaMail.jira@brutus>
Date: Wed, 4 Mar 2009 16:43:56 -0800 (PST)
From: "Jason Rutherglen (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Updated: (LUCENE-1516) Integrate IndexReader with
 IndexWriter
In-Reply-To: <2023665220.1231549319500.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/LUCENE-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated LUCENE-1516:
-------------------------------------

    Attachment: LUCENE-1516.patch

* TestConcurrentMergeScheduler is passing

* A handful of other tests don't pass, looking into them next

* DirectoryIndexReader.open with a writer isn't necessary so the
related method is removed

* I'm not sure we need to write out the deletes of unused segment
readers because they are no longer used and so should not be
required. I realized the unused readers needs to also check with
the deletion policy?

* The TestConcurrentMergeScheduler. testNoWaitClose exception is
caused by the way .del files are created when SR.commitChanges is
called, altering the SR segmentinfo without IFD.incRefing the .del
file. The solution, manually incref the .del file after the pool
commits on the SRs. Before this process was handled transparently by
IW.checkpoint. Also the other issue was the SegmentInfo files
variable was created before the delGen change, then the .del file was
not being returned by SI.files(). With the .del file being returned
the IFD related exception went away.

* SegmentInfoKey is removed, SegmentInfo is used as the key for
SegmentReaderPool.get

> You commented out the last part of commitMergedDeletes, that
actually saves the deletes. You need to instead get the reader for
the merged segment from the pool and hand it the new deletes.

I wasn't sure what you meant by this, in the patch the deletes are
copied into the merged reader. Do you mean instead the merged reader
should not be opened and instead the deletes file needs to be written
to?

> We lost the cutover to using the already computed docMap in
commitMergedDeletes?

SegmentMerger wasn't always returning the docMap so I stopped using
it.  

> Integrate IndexReader with IndexWriter 
> ---------------------------------------
>
>                 Key: LUCENE-1516
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1516
>             Project: Lucene - Java
>          Issue Type: Improvement
>    Affects Versions: 2.4
>            Reporter: Jason Rutherglen
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.9
>
>         Attachments: LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch, LUCENE-1516.patch
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> The current problem is an IndexReader and IndexWriter cannot be open
> at the same time and perform updates as they both require a write
> lock to the index. While methods such as IW.deleteDocuments enables
> deleting from IW, methods such as IR.deleteDocument(int doc) and
> norms updating are not available from IW. This limits the
> capabilities of performing updates to the index dynamically or in
> realtime without closing the IW and opening an IR, deleting or
> updating norms, flushing, then opening the IW again, a process which
> can be detrimental to realtime updates. 
> This patch will expose an IndexWriter.getReader method that returns
> the currently flushed state of the index as a class that implements
> IndexReader. The new IR implementation will differ from existing IR
> implementations such as MultiSegmentReader in that flushing will
> synchronize updates with IW in part by sharing the write lock. All
> methods of IR will be usable including reopen and clone. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org