Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 5 Sep 2016 08:37:20 +0000 (UTC)
From: "SammiChen (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12856613.1439817830000.490569.1473064640758@Atlassian.JIRA>
In-Reply-To: <JIRA.12856613.1439817830000@Atlassian.JIRA>
References: <JIRA.12856613.1439817830000@Atlassian.JIRA> <JIRA.12856613.1439817830826@arcas>
Subject: [jira] [Commented] (HDFS-8901) Use ByteBuffer in striping
 positional read
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 05 Sep 2016 08:37:22 -0000


    [ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464454#comment-15464454 ] 

SammiChen commented on HDFS-8901:
---------------------------------

Thanks Zhe!  Thanks Kai!

bq.  Why do we need to flip? In the previous code, arraycopy just starts copying from the current position of result.

I changed {{readAll(ByteBuffer buf, int len)}} implementation. In Previously implementation, position of {{buf}} will not be advanced by {{len}} after the all. I think it is not a traditional way when ByteBuffer parameter is used. SO I changed it. Now {{buf}} will be advanced by {{len}} after the call. That's why {{result}} should flip before read. 

bq.   Could you also explain why TestPread and TestSnapshotFileLength failed for v17 and v18 patches respectively?

Previously user input buffer is slice()ed in pread, and then the sliced buffer is passed into the callee.  The sliced buffer has its independent position and length.  This piece of code is optimized in pread, get rid of the slice(), directly pass the user input buffer to callee. So user buffer's position and length need to be handled carefully now.  TestPread and TestSnapshotFileLength failed because user buffer's position and read/write length doesn't took cared correctly in some execution path. 
    
bq.   Need to be removed:
Oops, will handle it. 

Based on our current discussion, I will reuse the approach in v14 patch to handle the readAll logic.


> Use ByteBuffer in striping positional read
> ------------------------------------------
>
>                 Key: HDFS-8901
>                 URL: https://issues.apache.org/jira/browse/HDFS-8901
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: SammiChen
>         Attachments: HDFS-8901-v10.patch, HDFS-8901-v17.patch, HDFS-8901-v18.patch, HDFS-8901-v19.patch, HDFS-8901-v2.patch, HDFS-8901-v3.patch, HDFS-8901-v4.patch, HDFS-8901-v5.patch, HDFS-8901-v6.patch, HDFS-8901-v7.patch, HDFS-8901-v8.patch, HDFS-8901-v9.patch, HDFS-8901.v11.patch, HDFS-8901.v12.patch, HDFS-8901.v13.patch, HDFS-8901.v14.patch, HDFS-8901.v15.patch, HDFS-8901.v16.patch, initial-poc.patch
>
>
> Native erasure coder prefers to direct ByteBuffer for performance consideration. To prepare for it, this change uses ByteBuffer through the codes in implementing striping position read. It will also fix avoiding unnecessary data copying between striping read chunk buffers and decode input buffers.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org