flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kurt Young (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-6309) Memory consumer weights should be calculated in job vertex level
Date Mon, 17 Apr 2017 01:28:42 GMT
Kurt Young created FLINK-6309:
---------------------------------

             Summary: Memory consumer weights should be calculated in job vertex level
                 Key: FLINK-6309
                 URL: https://issues.apache.org/jira/browse/FLINK-6309
             Project: Flink
          Issue Type: Improvement
          Components: Optimizer
            Reporter: Kurt Young


Currently in {{PlanFinalizer}}, we travel the whole job vertexes to calculate the memory consumer
weights, and then assign the weights for each job vertex. In a case of a large job graph,
e.g. with multiple joins, group reduces, the consumer weights will be high and the usable
memory for each job vertex will be very low. 
I think it makes more sense to calculate the memory consumer weights in job vertex level (after
chaining) to maximize the memory utility.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message