hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Broberg (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8258) Add interfaces for compression codecs to use direct byte buffers
Date Sat, 07 Apr 2012 01:07:19 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13249108#comment-13249108
] 

Tim Broberg commented on HADOOP-8258:
-------------------------------------

It's certainly trying to address the same thing, but then I haven't addressed the (arguably
more important) stream layer at all yet.

I'm not proud, feel free to kill whichever one you think is the weakling, but do please review
the proposed interface. I'm actually writing some code to it now, so any weaknesses you can
find in the interface sooner rather than later would be appreciated.
                
> Add interfaces for compression codecs to use direct byte buffers
> ----------------------------------------------------------------
>
>                 Key: HADOOP-8258
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8258
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: io, native, performance
>    Affects Versions: 3.0.0
>            Reporter: Todd Lipcon
>
> Currently, the codec interface only provides input/output functions based on byte arrays.
Given that most of the codecs are implemented in native code, this necessitates two extra
copies - one to copy the input data to a direct buffer, and one to copy the output data back
to a byte array. We should add interfaces to Decompressor/Compressor that can work directly
with direct byte buffers to avoid these copies.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message