flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1139) HadoopOutputFormat fails for DOP > 1
Date Fri, 21 Nov 2014 11:16:33 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220781#comment-14220781
] 

ASF GitHub Bot commented on FLINK-1139:
---------------------------------------

Github user StephanEwen commented on the pull request:

    https://github.com/apache/incubator-flink/pull/173#issuecomment-63957071
  
    This adds a hook at a very sensitive point. There one blocking issue that can cause the
job to hang.
    
    I would also not merge this without a test where the `finalizeGlobal()` method throws
an exception. That way we can be sure that the system handles such failures properly.


> HadoopOutputFormat fails for DOP > 1
> ------------------------------------
>
>                 Key: FLINK-1139
>                 URL: https://issues.apache.org/jira/browse/FLINK-1139
>             Project: Flink
>          Issue Type: Bug
>          Components: Hadoop Compatibility
>            Reporter: Fabian Hueske
>            Assignee: Fabian Hueske
>
> HadoopOutputFormat fails for DOP > 1. The reason is that job finalization (removal
of the temp directory) is done after the first task finishes.
> All other tasks will fail in that moment, since they cannot continue to write their data
to the temp directory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message