hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-13715) Add isErasureCoded() API to FileStatus class
Date Mon, 20 Mar 2017 22:36:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15933736#comment-15933736
] 

Manoj Govindassamy edited comment on HADOOP-13715 at 3/20/17 10:35 PM:
-----------------------------------------------------------------------

Thanks for the review [~stevel@apache.org]. Please take a look at the attached v02 patch.

bq. FileStatus.toString() needs to include the EC status.
Included erasure coded details in FileStatus.toString()

bq. There's enough assertFalse(fs.getFileStatus(dir).isErasureCoded()) and assertTrue that
they could be pulled out into a method with better diags
Created ContractTestUtils #assertErasureCoded and #assertNotErasureCoded and made the tests
to invoke this helper test util. 

bq. The filesystem specification doesn't have any coverage of erasure coding or this bit...
Updated {{filesystem.md}} to include Erasure Coding details in {{getFileStatus(p)}} specification.
Followed the encryption model in extending the result structure and a short description on
the same. Any suggestions on how to further improve this spec is highly welcome. 


was (Author: manojg):
Thanks for the review [~stevel@apache.org]. Please take a look at the attached v02 patch.

bq. FileStatus.toString() needs to include the EC status.
Included erasure coded details in FileStatus.toString()

bq. There's enough assertFalse(fs.getFileStatus(dir).isErasureCoded()) and assertTrue that
they could be pulled out into a method with better diags
Created ContractTestUtils #assertErasureCoded and #assertNotErasureCoded and made the tests
to invoke this helper test util. 

b2. The filesystem specification doesn't have any coverage of erasure coding or this bit...
Updated {{filesystem.md}} to include Erasure Coding details in {{getFileStatus(p)}} specification.
Followed the encryption model in extending the result structure and a short description on
the same. Any suggestions on how to further improve this spec is highly welcome. 

> Add isErasureCoded() API to FileStatus class
> --------------------------------------------
>
>                 Key: HADOOP-13715
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13715
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Wei-Chiu Chuang
>            Assignee: Manoj Govindassamy
>            Priority: Blocker
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HADOOP-13715.01.patch, HADOOP-13715.02.patch
>
>
> Per the discussion in [HDFS-10971|https://issues.apache.org/jira/browse/HDFS-10971?focusedCommentId=15567108&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15567108]
I would like to add a new API {{isErasureCoded()}} to {{FileStatus}} so that tools and downstream
applications can tell if it needs to treat a file differently.
> Hadoop tools that can benefit from this effort include: distcp and teragen/terasort.
> Downstream applications such as flume or hbase may also benefit from it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message