hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liran Oz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.
Date Mon, 20 Jun 2016 15:26:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339708#comment-15339708
] 

Liran Oz commented on HDFS-8668:
--------------------------------

Hi Kai,
I tried to use the native EC coder with hadoop-3.0.0-alpha1. so i applied the relevant patch
from HADOOP-11540 but i have noticed that every encode/decode operation requires additional
copy because the byteBuffer were not direct.
As i understand, HDFS-8668 patch should allocate direct ByteBuffers for the native coder -
but i did not managed to apply it.
Thank you for your response.
Liran.

> Erasure Coding: revisit buffer used for encoding and decoding.
> --------------------------------------------------------------
>
>                 Key: HDFS-8668
>                 URL: https://issues.apache.org/jira/browse/HDFS-8668
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Yi Liu
>            Assignee: Kai Zheng
>         Attachments: HDFS-8668-v1.patch, HDFS-8668-v2.patch
>
>
> For encoding and decoding buffers, currently some places use java heap ByteBuffer,  some
use direct byteBUffer, and some use java byte array.  If the coder implementation is native,
we should use direct ByteBuffer. This jira is to  revisit all encoding/decoding buffers and
improve them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message