Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Message-ID: <8213286.154701280910856091.JavaMail.jira@thor>
Date: Wed, 4 Aug 2010 04:34:16 -0400 (EDT)
From: "Shai Erera (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-2574) Optimize copies between IndexInput
 and Output
In-Reply-To: <31470353.51651280344635760.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12895181#action_12895181 ] 

Shai Erera commented on LUCENE-2574:
------------------------------------

Seems like it was left out. I'd like to add this fix, could you please review:

{code}
Index: lucene/src/java/org/apache/lucene/index/CompoundFileReader.java
===================================================================
--- lucene/src/java/org/apache/lucene/index/CompoundFileReader.java	(revision 982137)
+++ lucene/src/java/org/apache/lucene/index/CompoundFileReader.java	(working copy)
@@ -310,6 +310,11 @@
           // If there are more bytes left to copy, delegate the copy task to the
           // base IndexInput, in case it can do an optimized copy.
           if (numBytes > 0) {
+            long start = getFilePointer();
+            if (start + numBytes > length) {
+              throw new IOException("read past EOF");
+            }
+            base.seek(fileOffset + start);
             base.copyBytes(out, numBytes);
           }
         }
{code}

> Optimize copies between IndexInput and Output
> ---------------------------------------------
>
>                 Key: LUCENE-2574
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2574
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Shai Erera
>            Assignee: Shai Erera
>             Fix For: 3.1, 4.0
>
>         Attachments: LUCENE-2574.patch, LUCENE-2574.patch, LUCENE-2574.patch
>
>
> We've created an optimized copy of files from Directory to Directory. We've also optimized copyBytes recently. However, we're missing the opposite side of the copy - from IndexInput to Output. I'd like to mimic the FileChannel API by having copyTo on IndexInput and copyFrom on IndexOutput. That way, both sides can optimize the copy process, depending on the type of the IndexInput/Output that they need to copy to/from.
> FSIndexInput/Output can use FileChannel if the two are FS types. RAMInput/OutputStream can copy to/from the buffers directly, w/o going through intermediate ones. Actually, for RAMIn/Out this might be a big win, because it doesn't care about the type of IndexInput/Output given - it just needs to copy to its buffer directly.
> If we do this, I think we can consolidate all Dir.copy() impls down to one (in Directory), and rely on the In/Out ones to do the optimized copy. Plus, it will enable someone to do optimized copies between In/Out outside the scope of Directory.
> If this somehow turns out to be impossible, or won't make sense, then I'd like to optimize RAMDirectory.copy(Dir, src, dest) to not use an intermediate buffer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org