hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12924) Add default coder key for creating raw coders
Date Thu, 17 Mar 2016 01:07:33 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15198533#comment-15198533
] 

Kai Zheng commented on HADOOP-12924:
------------------------------------

Thanks for the discussions, Zhe and Rui.
bq. Kai mentioned that new RS Java coder and legacy RS coder from HDFS-RAID are not interoperable.
... they should be treated as 2 different policies rather than 2 different implementations?
Yes right, but I'm not sure we would have the additional policy for the legacy coder. I believe
the old coder is only useful for migrating old data from old systems (it's possible because
the coder resides in hadoop common side and not tightly coupled with any HDFS version), but
the migrating tool may not likely go into the current HDFS-EC implementation. So I suggest
we still use the current configuration key for the RS coder, to simplify the current HDFS
usage and also the HH coder implementation. In the feature user guide and doc, we should have
one section about erasure coders and how to configure/use them. It should explicitly mention
the legacy coder isn't compatible with the main RS coders (new Java coder, and ISA-L based
native coder). In fact, anytime users change the configured codec or coder, they should be
aware of the properties of these coders to understand which one is the right one to be used.


> Add default coder key for creating raw coders
> ---------------------------------------------
>
>                 Key: HADOOP-12924
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12924
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Rui Li
>            Assignee: Rui Li
>            Priority: Minor
>         Attachments: HADOOP-12924.1.patch
>
>
> As suggested [here|https://issues.apache.org/jira/browse/HADOOP-12826?focusedCommentId=15194402&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15194402].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message