hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sunil Govind <sunil.gov...@gmail.com>
Subject Re: how to add a shareable node label?
Date Fri, 07 Oct 2016 15:50:25 GMT
HI Frank

Thanks for the details.

I am not quite sure if I understood you problem correctly. I think you are
looking for a solution to ensure that Job_A will have to run as per its
schedule w/o any delay. Meantime you also do not want to waste resources on
those high end machine where Job_A is running.

I think you still need node label exclusivity here since there is h/w
dependency. But if you have 2 queues' which are shared to use "labelA"
here, then always "Job_A" can be planned to run in that queue, say
"queueA". Other jobs could be run in "queueB" here. So if you tune
capacities and if preemption is enabled per queue level, overutilized
resources used by "queueB" could be preempted for "Job_A".

But if your sharable jobs are like some linux jobs which should not be
preempted, then this may be only a half solution.


On Fri, Oct 7, 2016 at 7:36 AM Frank Luo <jluo@merkleinc.com> wrote:


You confirmed my understanding. I got the understanding by reading the docs
and haven’t really tried 2.8 or 3.0-alphal.

My situation is that I am in a multi-tenant env, and  got several very
powerful machines with expensive licenses to run a particular linux job,
let’s say Job_A. But the job is executed infrequently, so I want to let
other jobs to use the machines when Job_A is not running. In the meaning
time, I am not powerful enough to force all other jobs to be preemptable.
As matter of fact, I know they have Hadoop jobs inserting into sql-server,
or just pure linux jobs that are not preemptable in nature. So preempt jobs
is not an option for me.

I hope it makes sense.


*From:* Sunil Govind [mailto:sunil.govind@gmail.com]
*Sent:* Thursday, October 06, 2016 2:15 PM

*To:* Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org
*Subject:* Re: how to add a shareable node label?

HI Frank

Ideally those containers will be preempted if there are unsatisfied demand
for "configured label".

I could explain this:

"labelA" has few empty resources.  All nodes under "default" label is used.
Hence a new application which is submitted to "default" label has to wait.
But if "labelA" is non-exclusive and there are some free resources, this
new application can run on "labelA".

However if there are some more new apps submitted to "labelA", and if there
are no more resources available in "labelA", then it may preempt containers
from the app which was sharing containers earlier.

May be you could share some more information so tht it may become more
clear. Also I suppose you are running this in hadoop 3 alpha1 release.
please correct me if I m wrong.



On Thu, Oct 6, 2016 at 9:44 PM Frank Luo <jluo@merkleinc.com> wrote:

Thanks Sunil.

Ø  3. If there is any future ask for those resources , we will preempt the
non labeled apps and give them back to labeled apps.

Unfortunately, I am still not able to use it, because of the preemptive
behavior. The jobs that steals labelled resources are not preemptable, and
I’d rather waiting instead of killing.

*From:* Sunil Govind [mailto:sunil.govind@gmail.com]
*Sent:* Thursday, October 06, 2016 1:59 AM

*To:* Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org

*Subject:* Re: how to add a shareable node label?

Hi Frank

I think as of today this is not possible. You could try and experience the
"non-exlusive" feature of node-label which will officially come in 2.8
soon. Or you can try it in "Hadoop 3 alpha1" release too if its fine to
check. YARN-3214 <https://issues.apache.org/jira/browse/YARN-3214> has the
details for the nodelabel sharing concept.



On Wed, Oct 5, 2016 at 8:14 PM Frank Luo <jluo@merkleinc.com> wrote:

Sunil, thanks for responding.

So is there any way to dedicate one kind of jobs to certain machines, then
having those machines be shared if no dedicated job running?

*From:* Sunil Govind [mailto:sunil.govind@gmail.com]
*Sent:* Wednesday, October 05, 2016 12:50 AM
*To:* Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org;

*Subject:* Re: how to add a shareable node label?

Hi Frank,

As far as I checked, all labels are "exclusive" in 2.7. In upcoming 2.8
release, we can get "non-exclusive" or sharable node labels.



On Wed, Oct 5, 2016 at 8:40 AM Frank Luo <jluo@merkleinc.com> wrote:

I am using Hadoop 2.7.3, when I run:

$ yarn rmadmin -addToClusterNodeLabels "Label1(exclusive=false)"

I got an error as:

… addToClusterNodeLabels: java.io.IOException: label name should only
contains {0-9, a-z, A-Z, -, _} and should not started with {-,_}

If I just use “Label1”, it will work fine, but I want a shareable one.

Anyone knows a better way to do it?

*Access the Q2 2016 Digital Marketing Report for a fresh set of trends and
benchmarks in digital marketing*

*Download our latest report titled “The Case for Change: Exploring the
Myths of Customer-Centric Transformation” *

This email and any attachments transmitted with it are intended for use by
the intended recipient(s) only. If you have received this email in error,
please notify the sender immediately and then delete it. If you are not the
intended recipient, you must not keep, use, disclose, copy or distribute
this email without the author’s prior permission. We take precautions to
minimize the risk of transmitting software viruses, but we advise you to
perform your own virus checks on any attachment to this message. We cannot
accept liability for any loss or damage caused by software viruses. The
information contained in this communication may be confidential and may be
subject to the attorney-client privilege.

View raw message