hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carlo Curino (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4359) Update LowCost agents logic to take advantage of YARN-4358
Date Sat, 25 Jun 2016 01:21:16 GMT

    [ https://issues.apache.org/jira/browse/YARN-4359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348985#comment-15348985
] 

Carlo Curino commented on YARN-4359:
------------------------------------

[~imenache] thanks for updating the patch. It looks generally good, a couple of nits:

1) in {{StageAllocatorLowCostAligned}} near line 254 you commented out previous code, where
I think you can simply remove it.
2) you could/should use the expect exception when tests are expected to fail.
3) it would be good to defend against negative values in {{StageAllocatorLowCostAligned.getDurationInterval}}.
You subtract planModifications from netRLEs and things might go negative (unlikely but possible).
The rest of your code doesn't seem to protect against that. Maybe using {{subtractTestNegative}}?
Or manually checking the "value" of the Resource is >0. 
4) I believe I follow the changes and they look fine, but since there is lots going on in
this patch, can you run this in a live cluster, and check everything flows well (both for
Aligned and Greedy agents)?

I am ok with the checkstyles, there are only 2 "number of input parameters", one that was
already there, and the other one is ok I think.

[~subru] can you check this out as well.

Thanks,
Carlo

> Update LowCost agents logic to take advantage of YARN-4358
> ----------------------------------------------------------
>
>                 Key: YARN-4359
>                 URL: https://issues.apache.org/jira/browse/YARN-4359
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler, fairscheduler, resourcemanager
>            Reporter: Carlo Curino
>            Assignee: Ishai Menache
>         Attachments: YARN-4359.0.patch, YARN-4359.3.patch, YARN-4359.4.patch, YARN-4359.5.patch,
YARN-4359.6.patch, YARN-4359.7.patch, YARN-4359.8.patch
>
>
> Given the improvements of YARN-4358, the LowCost agent should be improved to leverage
this, and operate on RLESparseResourceAllocation (ideally leveraging the improvements of YARN-3454
to compute avaialable resources)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message