flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1139) HadoopOutputFormat fails for DOP > 1
Date Fri, 21 Nov 2014 11:16:33 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220781#comment-14220781

ASF GitHub Bot commented on FLINK-1139:

Github user StephanEwen commented on the pull request:

    This adds a hook at a very sensitive point. There one blocking issue that can cause the
job to hang.
    I would also not merge this without a test where the `finalizeGlobal()` method throws
an exception. That way we can be sure that the system handles such failures properly.

> HadoopOutputFormat fails for DOP > 1
> ------------------------------------
>                 Key: FLINK-1139
>                 URL: https://issues.apache.org/jira/browse/FLINK-1139
>             Project: Flink
>          Issue Type: Bug
>          Components: Hadoop Compatibility
>            Reporter: Fabian Hueske
>            Assignee: Fabian Hueske
> HadoopOutputFormat fails for DOP > 1. The reason is that job finalization (removal
of the temp directory) is done after the first task finishes.
> All other tasks will fail in that moment, since they cannot continue to write their data
to the temp directory.

This message was sent by Atlassian JIRA

View raw message