hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12924) Add default coder key for creating raw coders
Date Wed, 23 Mar 2016 06:28:25 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207952#comment-15207952

Zhe Zhang commented on HADOOP-12924:

Technically, any coder that yields new numerical results should be a new policy. Therefore
both the dummy coder and the legacy / HDFS-RAID RS coder should have their own policies.

[~drankye] I think the bottomline is that the file metadata (stored in the header) should
present HDFS-RAID and the new Java RS coder differently. Otherwise a reader won't be able
to correctly decode the data. I think we should just create a new policy for HDFS-RAID coder
(and one for the dummy coder). The concept of Reed-Solomon / {{RS}} is actually very broad.
Strictly speaking XOR is also a special case of RS. So I think our {{ErasureCodingPolicy}}
(and the name of each policy) should be explicit about what type of RS is being used.

> Add default coder key for creating raw coders
> ---------------------------------------------
>                 Key: HADOOP-12924
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12924
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Rui Li
>            Assignee: Rui Li
>            Priority: Minor
>         Attachments: HADOOP-12924.1.patch
> As suggested [here|https://issues.apache.org/jira/browse/HADOOP-12826?focusedCommentId=15194402&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15194402].

This message was sent by Atlassian JIRA

View raw message