hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6720) Support updating FPGA related constraint node label after FPGA device re-configuration
Date Fri, 07 Jul 2017 05:05:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077585#comment-16077585

Naganarasimha G R commented on YARN-6720:

Thanks [~tangzhankun]
bq.  a constraint label update after container finish to indicate docker image has been localized
is helpful to improve the scheduling. 
This was one of the improvements which i had in my mind, to automatically add labels to the
nodes for the localized Container images. We will develop it once YARN-3409 is in. This is
similar to the docker swarm functionality.

bq.  For instance, GPU handler for all different vendor might need to set a constraint "GPU_DOCKER_IMAGE_LOCALIZED:True/False"
to a node? FPGA handler for all vendor might need set "FPGA_IP_NAME:ipname"? If so, is it
a burden for end users to search and use these scheduling preference?
IIUC you are setting labels for "GPU_DOCKER_IMAGE_LOCALIZED:True/False" and/or "FPGA_IP_NAME:ipname",
so not many constraints (named newly as attribute ) right ? Can you elaborate more to understand
the use case ?

> Support updating FPGA related constraint node label after FPGA device re-configuration
> --------------------------------------------------------------------------------------
>                 Key: YARN-6720
>                 URL: https://issues.apache.org/jira/browse/YARN-6720
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Zhankun Tang
>         Attachments: Storing-and-Updating-extra-FPGA-resource-attributes-in-hdfs_v1.pdf
> In order to provide a global optimal scheduling for mutable FPGA resource, it seems an
easy and direct way to utilize constraint node labels(YARN-3409) instead of extending the
global scheduler(YARN-3926) to match both resource count and attributes.
> The rough idea is that the AM sets the constraint node label expression to request containers
on the nodes whose FPGA devices has the matching IP, and then NM resource handler update the
node constraint label if there's FPGA device re-configuration.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message