hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13715) Add isErasureCoded() API to FileStatus class
Date Mon, 20 Mar 2017 15:57:42 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932893#comment-15932893
] 

Steve Loughran commented on HADOOP-13715:
-----------------------------------------


* {{FileStatus.toString()}} needs to include the EC status. It'll be invaluable for assertions
and diagnostics
* The filesystem specification doesn't have any coverage of erasure coding or this bit. At
the very least it needs a mention in the FileStatus structure.
hich doesn't seem to have any explicit coverage except in the getFileStatus() call and in
invariants regarding consistency. Now would seem to be the time to
add more on the structure.
* There's enough {{assertFalse(fs.getFileStatus(dir).isErasureCoded())}} and assertTrue that
they could be pulled out into a method with better diags
 
{code}
 
assertErasureCoded(fs, path) {
 FileStatus s = fs.getFileStatus(path)
 assertTrue("Not erasure coded: " +s, s.isErasureCoded())
}
{code}
 
+ equivalent for assertNotErasureCoded. ContractTestUtils would be the obvious place for them.


> Add isErasureCoded() API to FileStatus class
> --------------------------------------------
>
>                 Key: HADOOP-13715
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13715
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Wei-Chiu Chuang
>            Assignee: Manoj Govindassamy
>            Priority: Blocker
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HADOOP-13715.01.patch
>
>
> Per the discussion in [HDFS-10971|https://issues.apache.org/jira/browse/HDFS-10971?focusedCommentId=15567108&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15567108]
I would like to add a new API {{isErasureCoded()}} to {{FileStatus}} so that tools and downstream
applications can tell if it needs to treat a file differently.
> Hadoop tools that can benefit from this effort include: distcp and teragen/terasort.
> Downstream applications such as flume or hbase may also benefit from it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message