hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wangda Tan <wheele...@gmail.com>
Subject Re: Node Label, partition or an attribute of node?
Date Thu, 25 Jun 2015 20:46:17 GMT
Hi Lei,
This is a very interesting topic, we have LOTS OF discussions with many
folks for resource vs. label.

The major concerns of not accepting user-provided resource type to resource
vectors are:
- It may simply doesn't work, dominate resource fairness cannot deal with
nodes with different set of resource vectors (some nodes have GPU and some
not).
- You cannot enforce it. For example, a node has 8 GPUs, and you allocate
to 5 processes, one gets 5 and the other one gets 3. But from YARN, you
cannot ensure nobody uses more than allocated.
- Adding new vectors needs lots of refactoring in protocol, scheduler, etc.
in YARN which is not figured out yet.

Currently, YARN doesn't have plan in short term accept user-provided
resource type to resource vector. But adding controllable resources like
IO/bandwidth, etc is on the road map.
So in the short term, for such non controllable resource, you can put them
to node label, you can enforce resource sharing via partition, or just a
binding requirement via attribute.

Thoughts?
WAngda

On Thu, Jun 25, 2015 at 1:27 PM, Lei Guo <leiguo@huawei.com> wrote:

> Hi, Wangda,
>
> Actually, it's the two types of node label causing the confusion for me.
> For a scheduling system, it's to find the best match between "resource" and
> "workload" via different "policies". Both node partition and node
> constraints are scheduling policies. The node label should be part of
> resource (as an attribute), and be used in two different policies. If we
> position the node label this way, there is only one type of node label.
> This is related to how we define/abstract "resource" in Yarn. Currently,
> only CPU/memory (extending with disk/network bandwidth) considered as
> resource for a node. As you mentioned in Summit slides, GPU could also be a
> resource. These are all consumable resources/attributes provided by node
> while node label is non-consumable resource/attribute provided by a node.
>
> Lei
>
> -----Original Message-----
> From: Wangda Tan [mailto:wheeleast@gmail.com]
> Sent: Thursday, June 25, 2015 3:51 PM
> To: yarn-dev@hadoop.apache.org
> Subject: Re: Node Label, partition or an attribute of node?
>
> Hi Lei,
> To clarify, node label has two types, one is node partition, which is
> supported already. Another is node constraint (you can also call it node
> attribute), which is still under design. Both of them are string typed.
>
> And node label (including partition and attribute), can be either
> per-application level and per-container level, this is supported by API
> already.
>
> And you can take a look at my summit slides:
> http://www.slideshare.net/Hadoop_Summit/node-labels-in-yarn-49792443.
> Which
> may help you better understand partition / constraint and relationship
> between them.
>
> Thanks,
> Wangda
>
> On Thu, Jun 25, 2015 at 12:34 PM, Lei Guo <leiguo@huawei.com> wrote:
>
> > The definition of label in YARN-796 is pretty clear,
> >
> > *         Node Label  label that describes a node. Node can have multiple
> > labels
> >
> > *         Label expression  logical combination of labels (using &&  and,
> > ||  or, !  not)
> > With this definition (without considering the implementation), a node
> > label could be treated as a Boolean type attribute of a node. Assuming
> the
> > administrator define label1, label2, label3 in the system, and associate
> > node1 with "label1, label2", this means the value of attribute label1 and
> > label2 for node1 is true, while the value of attribute label3 will be
> false
> > for node1.
> >
> > In current Yarn implementation, seems the resource partitioning has been
> > treated as the primary use case for node label, and some
> > design/implementation of node label mainly consider on the partitioning
> > case, even the multiple label support has been disabled in  YARN-2694. To
> > cover the workload resource request use case, YARN-3409 was proposed
> while
> > the description/example of constraint node label in YARN-3409 seems more
> > like a String type attribute for a node instead of Boolean type anymore.
> I
> > got confused about node label.
> >
> > From my view, node label is one attribute of a node, and it could be used
> > for different scheduling scenarios. No matter resource partitioning or
> job
> > scheduling constraints, a node label is just a label on a node. For
> > partitioning, we can have one object calling partition or resource group
> > representing the group of nodes with certain label; for workload
> scheduling
> > constraints purpose, we may want to extend the current application level
> > constraint to be container level constraint, for one application, the
> > master may request one type of resource while the slave request another
> > type of resource.
> >
> > Comments?
> >
> > Lei Guo
> > Senior Architect, Huawei Canada Research Centre
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message