hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-8446) Support of managing multi-dimensional resources
Date Mon, 25 Jun 2018 00:19:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-8446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Weiwei Yang updated YARN-8446:
------------------------------
    Description: 
To better support long running jobs and services, we need to extend YARN to support other
resources, such as disk, IP, port. Current resource types is not flexible enough to make this
work because it only supports COUNTABLE type which is single value.

Propose to extend resource types by adding two more general types, such as SET, MULTIDIMENSIONAL
(naming TBD). With schema like

*SET*:  a set of values
{noformat}
["10.100.0.1", "10.100.0.2"]
["9981", "9982", "9983"]
{noformat}
*MULTIDIMENSIONAL*: a set of values, each value can be a resource instance with multiple
values. 
{noformat}
[ disk1 : { attributes: { "type" : "SATA", "index" : 1 },  "size" : "500gb", "iops" : "1000"
},

  disk2 : { attributes: { "type" : "SSD", "index" : 2 },  "size" : "100gb", "iops" : "1000"
} ]
{noformat}
this way, we could support better resource management and isolations. The idea is to make
this as general as possible so we can easily support some other complex resources.

 

  was:
To better support long running jobs and services, we need to extend YARN to support other
resources, such as disk, IP, port. Current resource types is not flexible enough to make this
work because it only supports COUNTABLE type which is single value.

 

Propose to extend resource types by adding two more general types, such as SET, MULTIDIMENSIONAL
(naming TBD). With schema like

*SET*:  a set of values
{noformat}
["10.100.0.1", "10.100.0.2"]
["9981", "9982", "9983"]
{noformat}
*MULTIDIMENSIONAL*: a set of values, each value can be a resource instance with multiple
values. 
{noformat}
[ disk1 : { attributes: { "type" : "SATA", "index" : 1 },  "size" : "500gb", "iops" : "1000"
},

  disk2 : { attributes: { "type" : "SSD", "index" : 2 },  "size" : "100gb", "iops" : "1000"
} ]
{noformat}
this way, we could support better resource management and isolations. The idea is to make
this as general as possible so we can easily support some other complex resources.

 


> Support of managing multi-dimensional resources
> -----------------------------------------------
>
>                 Key: YARN-8446
>                 URL: https://issues.apache.org/jira/browse/YARN-8446
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Weiwei Yang
>            Priority: Major
>
> To better support long running jobs and services, we need to extend YARN to support other
resources, such as disk, IP, port. Current resource types is not flexible enough to make this
work because it only supports COUNTABLE type which is single value.
> Propose to extend resource types by adding two more general types, such as SET, MULTIDIMENSIONAL
(naming TBD). With schema like
> *SET*:  a set of values
> {noformat}
> ["10.100.0.1", "10.100.0.2"]
> ["9981", "9982", "9983"]
> {noformat}
> *MULTIDIMENSIONAL*: a set of values, each value can be a resource instance with multiple
values. 
> {noformat}
> [ disk1 : { attributes: { "type" : "SATA", "index" : 1 },  "size" : "500gb", "iops"
: "1000" },
>   disk2 : { attributes: { "type" : "SSD", "index" : 2 },  "size" : "100gb", "iops"
: "1000" } ]
> {noformat}
> this way, we could support better resource management and isolations. The idea is to
make this as general as possible so we can easily support some other complex resources.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message