hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10047) Add a directbuffer Decompressor API to hadoop
Date Wed, 30 Oct 2013 12:53:28 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gopal V updated HADOOP-10047:
-----------------------------

    Description: 
With the Zero-Copy reads in HDFS (HDFS-5260), it becomes important to perform all I/O operations
without copying data into byte[] buffers or other buffers which wrap over them.

This is a proposal for adding a DirectDecompressor interface to the io.compress, to indicate
codecs which want to surface the direct buffer layer upwards.

The implementation should work with direct heap/mmap buffers and cannot assume .array() availability.

  was:
With the Zero-Copy reads in HDFS (HDFS-5260), it becomes important to perform all I/O operations
without copying data into byte[] buffers or other buffers which wrap over them.

This is a proposal for adding new DirectCompressor and DirectDecompressor interfaces to the
io.compress, to indicate codecs which want to surface the direct buffer layer upwards.

The implementation may or may not copy the buffers passed in, but should work with direct
heap/mmap buffers and cannot assume .array() availability.


> Add a directbuffer Decompressor API to hadoop
> ---------------------------------------------
>
>                 Key: HADOOP-10047
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10047
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: io
>            Reporter: Gopal V
>            Assignee: Gopal V
>              Labels: compression
>         Attachments: DirectCompressor.html, DirectDecompressor.html, HADOOP-10047-redo-WIP.patch,
HADOOP-10047-WIP.patch, HADOOP-10047-with-tests.patch
>
>
> With the Zero-Copy reads in HDFS (HDFS-5260), it becomes important to perform all I/O
operations without copying data into byte[] buffers or other buffers which wrap over them.
> This is a proposal for adding a DirectDecompressor interface to the io.compress, to indicate
codecs which want to surface the direct buffer layer upwards.
> The implementation should work with direct heap/mmap buffers and cannot assume .array()
availability.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message