hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read
Date Mon, 20 Jun 2016 01:36:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338872#comment-15338872
] 

Kai Zheng commented on HDFS-8901:
---------------------------------

Thanks Bo for the update, and fixing of the test failure. 

The fix by implementing the {{read(ByteBuffer buf)}} method looks good, but I wonder if we
could remove the long deprecated class entirely at all. Will raise this separately.
{code}
diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/BlockReaderRemote.java
b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/BlockReaderRemote.java
index 22d4e23..d7c5da9 100644
--- a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/BlockReaderRemote.java
+++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/BlockReaderRemote.java
@@ -480,7 +480,21 @@ void sendReadResult(Peer peer, Status statusCode) {
 
   @Override
   public int read(ByteBuffer buf) throws IOException {
-    throw new UnsupportedOperationException("readDirect unsupported in BlockReaderRemote");
+    int toRead = buf.remaining();
+    int nRead;
+    if (buf.isDirect()) {
+      byte[] bytes = new byte[toRead];
+      nRead = read(bytes, 0, toRead);
+      if (nRead > 0) {
+        buf.put(bytes, 0, nRead);
+      }
+    } else {
+      nRead = read(buf.array(), buf.arrayOffset(), toRead);
+      if (nRead > 0) {
+        buf.position(buf.position() + nRead);
+      }
+    }
+    return nRead;
   }
{code}

> Use ByteBuffer in striping positional read
> ------------------------------------------
>
>                 Key: HDFS-8901
>                 URL: https://issues.apache.org/jira/browse/HDFS-8901
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>         Attachments: HDFS-8901-v2.patch, HDFS-8901-v3.patch, HDFS-8901-v4.patch, HDFS-8901-v5.patch,
HDFS-8901-v6.patch, HDFS-8901-v7.patch, HDFS-8901-v8.patch, HDFS-8901-v9.patch, initial-poc.patch
>
>
> Native erasure coder prefers to direct ByteBuffer for performance consideration. To prepare
for it, this change uses ByteBuffer through the codes in implementing striping position read.
It will also fix avoiding unnecessary data copying between striping read chunk buffers and
decode input buffers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message