hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13715) Add isErasureCoded() API to FileStatus class
Date Mon, 20 Mar 2017 15:57:42 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932893#comment-15932893

Steve Loughran commented on HADOOP-13715:

* {{FileStatus.toString()}} needs to include the EC status. It'll be invaluable for assertions
and diagnostics
* The filesystem specification doesn't have any coverage of erasure coding or this bit. At
the very least it needs a mention in the FileStatus structure.
hich doesn't seem to have any explicit coverage except in the getFileStatus() call and in
invariants regarding consistency. Now would seem to be the time to
add more on the structure.
* There's enough {{assertFalse(fs.getFileStatus(dir).isErasureCoded())}} and assertTrue that
they could be pulled out into a method with better diags
assertErasureCoded(fs, path) {
 FileStatus s = fs.getFileStatus(path)
 assertTrue("Not erasure coded: " +s, s.isErasureCoded())
+ equivalent for assertNotErasureCoded. ContractTestUtils would be the obvious place for them.

> Add isErasureCoded() API to FileStatus class
> --------------------------------------------
>                 Key: HADOOP-13715
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13715
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Wei-Chiu Chuang
>            Assignee: Manoj Govindassamy
>            Priority: Blocker
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HADOOP-13715.01.patch
> Per the discussion in [HDFS-10971|https://issues.apache.org/jira/browse/HDFS-10971?focusedCommentId=15567108&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15567108]
I would like to add a new API {{isErasureCoded()}} to {{FileStatus}} so that tools and downstream
applications can tell if it needs to treat a file differently.
> Hadoop tools that can benefit from this effort include: distcp and teragen/terasort.
> Downstream applications such as flume or hbase may also benefit from it.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message