flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3256) Invalid execution graph cleanup for jobs with colocation groups
Date Wed, 20 Jan 2016 15:35:39 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108743#comment-15108743

ASF GitHub Bot commented on FLINK-3256:

Github user StephanEwen commented on the pull request:

    Thanks for fixing this, looks like a good issue!
    One suggestion for a change: Rather than collecting the CoLocationGroups in an extra set
in the ExecutionGraph, can you iterate over the ExecutionJobvertices and simply reset them
    I think each field less on the ExecutionGraph makes this simpler to maintain and more

> Invalid execution graph cleanup for jobs with colocation groups
> ---------------------------------------------------------------
>                 Key: FLINK-3256
>                 URL: https://issues.apache.org/jira/browse/FLINK-3256
>             Project: Flink
>          Issue Type: Bug
>          Components: Distributed Runtime
>            Reporter: Paris Carbone
>            Assignee: Paris Carbone
>            Priority: Blocker
> Currently, upon restarting an execution graph, we clean-up the colocation constraints
for each group present in an ExecutionJobVertex respectively.
> This can lead to invalid reconfiguration upon a restart or any other activity that relies
on state cleanup of the execution graph. For example, upon restarting a DataStream job with
iterations the following steps are executed:
> 1) IterationSource colgroup constraints are reset
> 2) IterationSource execution vertices reset and create new colocation constraints
> 3) IterationSink colgroup constraints are reset
> 4) IterationSink execution vertices reset and create different colocation constraints.
> This can be trivially fixed by reseting colocation groups independently from ExecutionJobVertices,
thus, updating them once per reconfiguration.

This message was sent by Atlassian JIRA

View raw message