flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-3787) Yarn client does not report unfulfillable container constraints
Date Tue, 19 Apr 2016 12:46:25 GMT
Till Rohrmann created FLINK-3787:
------------------------------------

             Summary: Yarn client does not report unfulfillable container constraints
                 Key: FLINK-3787
                 URL: https://issues.apache.org/jira/browse/FLINK-3787
             Project: Flink
          Issue Type: Improvement
          Components: YARN Client
    Affects Versions: 1.1.0
            Reporter: Till Rohrmann
            Priority: Minor


If the number of virtual cores for a Yarn container is not fulfillable, then the {{TaskManager}}
won't be started. This is only reported in the logs but not in the {{FlinkYarnClient}}. Thus,
the user will see a started {{JobManager}} with no connected {{TaskManagers}}. Since the log
aggregation is only available after the Yarn job has been stopped, there is no easy way for
the user to detect what's going on.

This problem is aggravated by the fact that the number of virtual cores is coupled to the
number of slots if no explicit value has been set for the virtual cores. Therefore, it might
happen that the Yarn deployment fails because of the virtual cores even though the user has
never set a value for them (the user might even not know about the virtual cores).

I think it would be good to check if the virtual cores constraint is fulfillable. If not,
then the user should receive a clear message that the Flink cluster cannot be deployed (similar
to the memory constraints).  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message