lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Rutherglen (JIRA)" <j...@apache.org>
Subject [jira] Updated: (LUCENE-1738) IndexWriter.addIndexes without syncing
Date Wed, 15 Jul 2009 19:18:14 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jason Rutherglen updated LUCENE-1738:
-------------------------------------

    Attachment: LUCENE-1738.patch

Very basic start at the patch.  The not IW.dir check is removed in the DirectoryReader ctor
called by readerPool.  This conflicts with the way addIndexes currently works.  I guess we
could add a parameter to segmentInfo indicating it's ok to include the segmentInfo in getReader?
  

* Added IW.addIndexesNoSync which doesn't stop indexing during the method, nor does it synchronously
copy the indexes over.  The new indexes are scheduled as merges.  

* commit and close call resolveExternalSegments.

* I think we'll want a boolean parameter that synchronously copies the indexes over but does
not start any merging.  This is for copying from a filesystem index.  In the replication use
case, we're adding ramDirs so we don't need to immediately merge/copy them over.

* Needs more unit tests

> IndexWriter.addIndexes without syncing
> --------------------------------------
>
>                 Key: LUCENE-1738
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1738
>             Project: Lucene - Java
>          Issue Type: Improvement
>    Affects Versions: 2.4.1
>            Reporter: Jason Rutherglen
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-1738.patch
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> When LUCENE-1313 is completed, it would be good to have a way to
> replicate segments from one IndexWriter to another.
> * Callback on successful flush (maybe for other events as well?)
> * Ability to access files for a segment (which would presumably
> be read from the IW ramdir), then copy them to a temporary
> serialized ramdir (or equivalent as ramdir uses extra space in
> blocks, whereas we'll already know the size of the files before
> we write them).
> * On the receiving end, we may be able to use
> addIndexesNoOptimize(Directory[]), however this would entail
> each directory having an extraneous segment_N file for each
> replicated update (so we may want another format). 
> * It will rely on having a new public version of SegmentInfo. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message