hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Luo <j...@merkleinc.com>
Subject RE: how to add a shareable node label?
Date Thu, 06 Oct 2016 16:14:05 GMT
Thanks Sunil.


Ø  3. If there is any future ask for those resources , we will preempt the non labeled apps
and give them back to labeled apps.

Unfortunately, I am still not able to use it, because of the preemptive behavior. The jobs
that steals labelled resources are not preemptable, and I’d rather waiting instead of killing.

From: Sunil Govind [mailto:sunil.govind@gmail.com]
Sent: Thursday, October 06, 2016 1:59 AM
To: Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org
Subject: Re: how to add a shareable node label?

Hi Frank
I think as of today this is not possible. You could try and experience the "non-exlusive"
feature of node-label which will officially come in 2.8 soon. Or you can try it in "Hadoop
3 alpha1" release too if its fine to check. YARN-3214<https://issues.apache.org/jira/browse/YARN-3214>
has the details for the nodelabel sharing concept.

Thanks
Sunil

On Wed, Oct 5, 2016 at 8:14 PM Frank Luo <jluo@merkleinc.com<mailto:jluo@merkleinc.com>>
wrote:
Sunil, thanks for responding.

So is there any way to dedicate one kind of jobs to certain machines, then having those machines
be shared if no dedicated job running?

From: Sunil Govind [mailto:sunil.govind@gmail.com<mailto:sunil.govind@gmail.com>]
Sent: Wednesday, October 05, 2016 12:50 AM
To: Frank Luo <jluo@merkleinc.com<mailto:jluo@merkleinc.com>>; user@hadoop.apache.org<mailto:user@hadoop.apache.org>;
user@yarn.apache.org<mailto:user@yarn.apache.org>

Subject: Re: how to add a shareable node label?

Hi Frank,

As far as I checked, all labels are "exclusive" in 2.7. In upcoming 2.8 release, we can get
"non-exclusive" or sharable node labels.

Thanks
Sunil

On Wed, Oct 5, 2016 at 8:40 AM Frank Luo <jluo@merkleinc.com<mailto:jluo@merkleinc.com>>
wrote:
I am using Hadoop 2.7.3, when I run:
$ yarn rmadmin -addToClusterNodeLabels "Label1(exclusive=false)"

I got an error as:

… addToClusterNodeLabels: java.io.IOException: label name should only contains {0-9, a-z,
A-Z, -, _} and should not started with {-,_}

If I just use “Label1”, it will work fine, but I want a shareable one.

Anyone knows a better way to do it?

Access the Q2 2016 Digital Marketing Report for a fresh set of trends and benchmarks in digital
marketing<http://www2.merkleinc.com/l/47252/2016-07-26/47gt7c>

Download our latest report titled “The Case for Change: Exploring the Myths of Customer-Centric
Transformation” <http://www2.merkleinc.com/l/47252/2016-08-04/4b9p7c>

This email and any attachments transmitted with it are intended for use by the intended recipient(s)
only. If you have received this email in error, please notify the sender immediately and then
delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or
distribute this email without the author’s prior permission. We take precautions to minimize
the risk of transmitting software viruses, but we advise you to perform your own virus checks
on any attachment to this message. We cannot accept liability for any loss or damage caused
by software viruses. The information contained in this communication may be confidential and
may be subject to the attorney-client privilege.

Access the Q2 2016 Digital Marketing Report for a fresh set of trends and benchmarks in digital
marketing<http://www2.merkleinc.com/l/47252/2016-07-26/47gt7c>

Download our latest report titled “The Case for Change: Exploring the Myths of Customer-Centric
Transformation” <http://www2.merkleinc.com/l/47252/2016-08-04/4b9p7c>

This email and any attachments transmitted with it are intended for use by the intended recipient(s)
only. If you have received this email in error, please notify the sender immediately and then
delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or
distribute this email without the author’s prior permission. We take precautions to minimize
the risk of transmitting software viruses, but we advise you to perform your own virus checks
on any attachment to this message. We cannot accept liability for any loss or damage caused
by software viruses. The information contained in this communication may be confidential and
may be subject to the attorney-client privilege.

Access the Q2 2016 Digital Marketing Report for a fresh set of trends and benchmarks in digital
marketing<http://www2.merkleinc.com/l/47252/2016-07-26/47gt7c>

Download our latest report titled “The Case for Change: Exploring the Myths of Customer-Centric
Transformation” <http://www2.merkleinc.com/l/47252/2016-08-04/4b9p7c>

This email and any attachments transmitted with it are intended for use by the intended recipient(s)
only. If you have received this email in error, please notify the sender immediately and then
delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or
distribute this email without the author’s prior permission. We take precautions to minimize
the risk of transmitting software viruses, but we advise you to perform your own virus checks
on any attachment to this message. We cannot accept liability for any loss or damage caused
by software viruses. The information contained in this communication may be confidential and
may be subject to the attorney-client privilege.
Mime
View raw message