lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias Seidel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-2574) Optimize copies between IndexInput and Output
Date Thu, 07 Apr 2011 10:08:06 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13016794#comment-13016794
] 

Matthias Seidel commented on LUCENE-2574:
-----------------------------------------

Uhm, what happened to these optimizations to FSIndexOutput regarding the direct copying from
FSIndexInput via its FileChannel? Can't find a trace of it anywhere in the 3.1 release or
in the trunk. :/

> Optimize copies between IndexInput and Output
> ---------------------------------------------
>
>                 Key: LUCENE-2574
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2574
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Store
>            Reporter: Shai Erera
>            Assignee: Shai Erera
>             Fix For: 3.1, 4.0
>
>         Attachments: LUCENE-2574.patch, LUCENE-2574.patch, LUCENE-2574.patch
>
>
> We've created an optimized copy of files from Directory to Directory. We've also optimized
copyBytes recently. However, we're missing the opposite side of the copy - from IndexInput
to Output. I'd like to mimic the FileChannel API by having copyTo on IndexInput and copyFrom
on IndexOutput. That way, both sides can optimize the copy process, depending on the type
of the IndexInput/Output that they need to copy to/from.
> FSIndexInput/Output can use FileChannel if the two are FS types. RAMInput/OutputStream
can copy to/from the buffers directly, w/o going through intermediate ones. Actually, for
RAMIn/Out this might be a big win, because it doesn't care about the type of IndexInput/Output
given - it just needs to copy to its buffer directly.
> If we do this, I think we can consolidate all Dir.copy() impls down to one (in Directory),
and rely on the In/Out ones to do the optimized copy. Plus, it will enable someone to do optimized
copies between In/Out outside the scope of Directory.
> If this somehow turns out to be impossible, or won't make sense, then I'd like to optimize
RAMDirectory.copy(Dir, src, dest) to not use an intermediate buffer.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message