hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SammiChen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read
Date Fri, 26 Aug 2016 05:38:22 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438489#comment-15438489

SammiChen commented on HDFS-8901:

Hi Zhe,

Thanks for your great effort to review the patch. Very good suggestions. Here is my thoughts.

1. the piece of code will be moved to BlockReaderUtil, create a new readAll function for it.

2. will be handled.
3. Yes. we are planning to add a ByteBuffer version read API
4. good question, I'm double checking the code. And will share my finding later.
5. will be handled.
6. Every StripingChunk will have either a ChunkByteBuffer or a ByteBuffer to track different
buffer source. ChunkByteBuffer will be used when reusing user input buffer as the striping
read buffer during positional read. And ByteBuffer will be used when read using stateful strip
7. the memory copy is avoid in case chunk.useChunkBuffer is not true, that's an improvement.

> Use ByteBuffer in striping positional read
> ------------------------------------------
>                 Key: HDFS-8901
>                 URL: https://issues.apache.org/jira/browse/HDFS-8901
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: SammiChen
>         Attachments: HDFS-8901-v10.patch, HDFS-8901-v2.patch, HDFS-8901-v3.patch, HDFS-8901-v4.patch,
HDFS-8901-v5.patch, HDFS-8901-v6.patch, HDFS-8901-v7.patch, HDFS-8901-v8.patch, HDFS-8901-v9.patch,
HDFS-8901.v11.patch, HDFS-8901.v12.patch, HDFS-8901.v13.patch, HDFS-8901.v14.patch, initial-poc.patch
> Native erasure coder prefers to direct ByteBuffer for performance consideration. To prepare
for it, this change uses ByteBuffer through the codes in implementing striping position read.
It will also fix avoiding unnecessary data copying between striping read chunk buffers and
decode input buffers.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message