hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12473) distcp's ignoring failures should be mutually exclusive with the atomic option
Date Fri, 30 Oct 2015 18:19:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14983007#comment-14983007
] 

Mingliang Liu commented on HADOOP-12473:
----------------------------------------

Thanks to your comment [~Prabhu Joseph]. I think the {{ignoreFailures}} is to make the copy
mapper ignore only {{RetriableFileCopyCommand.CopyReadException}} failures? Sorry I don't
have the context about MRv1.

> distcp's ignoring failures should be mutually exclusive with the atomic option
> ------------------------------------------------------------------------------
>
>                 Key: HADOOP-12473
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12473
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: tools/distcp
>    Affects Versions: 2.7.1
>            Reporter: Mingliang Liu
>            Priority: Critical
>             Fix For: 2.8.0
>
>
> In {{CopyMapper::handleFailure}}, the mapper handles failure and will ignore it if no
it's config key is on. Ignoring failures should be mutually exclusive with the atomic option
otherwise an incomplete dir is eligible for commit defeating the purpose.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message