hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mithun Radhakrishnan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12473) distcp's ignoring failures should be mutually exclusive with the atomic option
Date Mon, 21 Dec 2015 23:00:48 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067214#comment-15067214
] 

Mithun Radhakrishnan commented on HADOOP-12473:
-----------------------------------------------

[~jira.shegalov], [~liuml07], could you please explain the reasoning behind this fix?

If I understand correctly, Gera's orthogonal suggestion in HADOOP-12469 was to make {{-atomic}}
and {{-i}} mutually exclusive. The latest patch in this JIRA doesn't seem to address this
concern, AFAICT. It makes {{ignoreFailures}} an {{AtomicBoolean}}, which is not what Gera
was getting at, I believe.

Also, [~jira.shegalov], why do you recommend that {{-atomic}} and {{-i}} be mutually exclusive?
Aren't they orthogonal concerns? Why consider {{-atomic}} as incapable of ignoring copy-errors?



> distcp's ignoring failures should be mutually exclusive with the atomic option
> ------------------------------------------------------------------------------
>
>                 Key: HADOOP-12473
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12473
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: tools/distcp
>    Affects Versions: 2.7.1
>            Reporter: Mingliang Liu
>            Assignee: Mingliang Liu
>             Fix For: 2.8.0
>
>         Attachments: HADOOP-12473.000.patch, HADOOP-12473.001.patch, HADOOP-12473.002.patch
>
>
> In {{CopyMapper::handleFailure}}, the mapper handles failure and will ignore it if no
it's config key is on. Ignoring failures should be mutually exclusive with the atomic option
otherwise an incomplete dir is eligible for commit defeating the purpose.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message