hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13715) Add isErasureCoded() API to FileStatus class
Date Thu, 23 Mar 2017 01:25:42 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937518#comment-15937518
] 

Andrew Wang commented on HADOOP-13715:
--------------------------------------

Hi Manoj, thanks for the quick turnaround! I took a quick look and there's one thing I don't
understand, I see this in the modified testListStatus:

{code}
    if (proxyFs.exists(TestHdfsHelper.ERASURE_CODING_DIR)) {
      ContractTestUtils.assertNotErasureCoded(httpFS,
          TestHdfsHelper.ERASURE_CODING_DIR);
      ContractTestUtils.assertNotErasureCoded(httpFS,
          TestHdfsHelper.ERASURE_CODING_FILE);
    }
{code}

Shouldn't the second one be assertErasureCoded? i.e. the EC file should have the EC bit set?

It might also be cleaner to do this in a separate test like testEncryption and testFileAcls.

BTW I think we should file a follow-on JIRA to add the different FileStatus bits to FileStatus#toString
as Steve recommended, seems like it's a safe change as long as it's not being printed to the
shell.

> Add isErasureCoded() API to FileStatus class
> --------------------------------------------
>
>                 Key: HADOOP-13715
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13715
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Wei-Chiu Chuang
>            Assignee: Manoj Govindassamy
>            Priority: Blocker
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HADOOP-13715.01.patch, HADOOP-13715.02.patch, HADOOP-13715.03.patch,
HADOOP-13715.04.patch
>
>
> Per the discussion in [HDFS-10971|https://issues.apache.org/jira/browse/HDFS-10971?focusedCommentId=15567108&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15567108]
I would like to add a new API {{isErasureCoded()}} to {{FileStatus}} so that tools and downstream
applications can tell if it needs to treat a file differently.
> Hadoop tools that can benefit from this effort include: distcp and teragen/terasort.
> Downstream applications such as flume or hbase may also benefit from it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message