lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Busch (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-600) ParallelWriter companion to ParallelReader
Date Tue, 01 Sep 2009 02:47:32 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749740#action_12749740
] 

Michael Busch commented on LUCENE-600:
--------------------------------------

{quote}
I could inquire if you are truly open to it, but it sounds like you may be on your own path
to a quite similar thing.
{quote}

Well my goal is to get the best possible implementation of this feature into Lucene. Nothing
is set in stone yet. So you should feel free to suggest improvements. And if you think your
implementation is better or has details worth looking at it would be good if you could submit
your code.

> ParallelWriter companion to ParallelReader
> ------------------------------------------
>
>                 Key: LUCENE-600
>                 URL: https://issues.apache.org/jira/browse/LUCENE-600
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 2.1
>            Reporter: Chuck Williams
>            Priority: Minor
>         Attachments: ParallelWriter.patch
>
>
> A new class ParallelWriter is provided that serves as a companion to ParallelReader.
 ParallelWriter meets all of the doc-id synchronization requirements of ParallelReader, subject
to:
>     1.  ParallelWriter.addDocument() is synchronized, which might have an adverse effect
on performance.  The writes to the sub-indexes are, however, done in parallel.
>     2.  The application must ensure that the ParallelReader is never reopened inside
ParallelWriter.addDocument(), else it might find the sub-indexes out of sync.
>     3.  The application must deal with recovery from ParallelWriter.addDocument() exceptions.
 Recovery must restore the synchronization of doc-ids, e.g. by deleting any trailing document(s)
in one sub-index that were not successfully added to all sub-indexes, and then optimizing
all sub-indexes.
> A new interface, Writable, is provided to abstract IndexWriter and ParallelWriter.  This
is in the same spirit as the existing Searchable and Fieldable classes.
> This implementation uses java 1.5.  The patch applies against today's svn head.  All
tests pass, including the new TestParallelWriter.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message