hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sunil Govind <sunil.gov...@gmail.com>
Subject Re: how to add a shareable node label?
Date Tue, 11 Oct 2016 16:44:16 GMT
Hi Frank

Extremely sorry for the delay..

Yes, you are correct. Sharing feature of node label is not needed in your
case.
Existing node labels and a queue model could solve the problem.

Thanks
Sunil

On Fri, Oct 7, 2016 at 11:59 PM Frank Luo <jluo@merkleinc.com> wrote:

> That is correct, Sunil.
>
>
>
> Just to confirm,  the Node Labeling feature on 2.8 or 3.0 alpha won’t
> satisfy my need, right?
>
>
>
> *From:* Sunil Govind [mailto:sunil.govind@gmail.com]
> *Sent:* Friday, October 07, 2016 12:09 PM
>
>
> *To:* Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org
> *Subject:* Re: how to add a shareable node label?
>
>
>
> HI Frank
>
>
>
> In that case, preemption may not be needed. So over-utilizing resources of
> queueB will be running till it completes. Since queueA is under served,
> then any next free container could go to queueA which is for Job_A.
>
>
>
> Thanks
>
> Sunil
>
>
>
> On Fri, Oct 7, 2016 at 9:58 PM Frank Luo <jluo@merkleinc.com> wrote:
>
> Sunil,
>
>
>
> Your description pretty much matches my understanding. Except for “Job_A
> will have to run as per its schedule w/o any delay”. My situation is that
> Job_A can be delayed. As long as it runs in queueA, I am happy.
>
>
>
> Just as you said, processes normally running in queueB might not be
> preemptable. So if they overflow to queueA then got preempted, then that is
> not good.
>
>
>
> *From:* Sunil Govind [mailto:sunil.govind@gmail.com]
> *Sent:* Friday, October 07, 2016 10:50 AM
>
>
> *To:* Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org
>
> *Subject:* Re: how to add a shareable node label?
>
>
>
> HI Frank
>
>
>
> Thanks for the details.
>
>
>
> I am not quite sure if I understood you problem correctly. I think you are
> looking for a solution to ensure that Job_A will have to run as per its
> schedule w/o any delay. Meantime you also do not want to waste resources on
> those high end machine where Job_A is running.
>
>
>
> I think you still need node label exclusivity here since there is h/w
> dependency. But if you have 2 queues' which are shared to use "labelA"
> here, then always "Job_A" can be planned to run in that queue, say
> "queueA". Other jobs could be run in "queueB" here. So if you tune
> capacities and if preemption is enabled per queue level, overutilized
> resources used by "queueB" could be preempted for "Job_A".
>
>
>
> But if your sharable jobs are like some linux jobs which should not be
> preempted, then this may be only a half solution.
>
>
>
> Thanks
>
> Sunil
>
>
>
> On Fri, Oct 7, 2016 at 7:36 AM Frank Luo <jluo@merkleinc.com> wrote:
>
> Sunil,
>
>
>
> You confirmed my understanding. I got the understanding by reading the
> docs and haven’t really tried 2.8 or 3.0-alphal.
>
>
>
> My situation is that I am in a multi-tenant env, and  got several very
> powerful machines with expensive licenses to run a particular linux job,
> let’s say Job_A. But the job is executed infrequently, so I want to let
> other jobs to use the machines when Job_A is not running. In the meaning
> time, I am not powerful enough to force all other jobs to be preemptable.
> As matter of fact, I know they have Hadoop jobs inserting into sql-server,
> or just pure linux jobs that are not preemptable in nature. So preempt jobs
> is not an option for me.
>
>
>
> I hope it makes sense.
>
>
>
> Frank
>
>
>
> *From:* Sunil Govind [mailto:sunil.govind@gmail.com]
> *Sent:* Thursday, October 06, 2016 2:15 PM
>
>
> *To:* Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org
> *Subject:* Re: how to add a shareable node label?
>
>
>
> HI Frank
>
>
>
> Ideally those containers will be preempted if there are unsatisfied demand
> for "configured label".
>
>
>
> I could explain this:
>
> "labelA" has few empty resources.  All nodes under "default" label is
> used. Hence a new application which is submitted to "default" label has to
> wait. But if "labelA" is non-exclusive and there are some free resources,
> this new application can run on "labelA".
>
> However if there are some more new apps submitted to "labelA", and if
> there are no more resources available in "labelA", then it may preempt
> containers from the app which was sharing containers earlier.
>
>
>
> May be you could share some more information so tht it may become more
> clear. Also I suppose you are running this in hadoop 3 alpha1 release.
> please correct me if I m wrong.
>
>
>
> Thanks
>
> Sunil
>
>
>
> On Thu, Oct 6, 2016 at 9:44 PM Frank Luo <jluo@merkleinc.com> wrote:
>
> Thanks Sunil.
>
>
>
> Ø  3. If there is any future ask for those resources , we will preempt
> the non labeled apps and give them back to labeled apps.
>
>
>
> Unfortunately, I am still not able to use it, because of the preemptive
> behavior. The jobs that steals labelled resources are not preemptable, and
> I’d rather waiting instead of killing.
>
>
>
> *From:* Sunil Govind [mailto:sunil.govind@gmail.com]
> *Sent:* Thursday, October 06, 2016 1:59 AM
>
>
> *To:* Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org
>
> *Subject:* Re: how to add a shareable node label?
>
>
>
> Hi Frank
>
> I think as of today this is not possible. You could try and experience the
> "non-exlusive" feature of node-label which will officially come in 2.8
> soon. Or you can try it in "Hadoop 3 alpha1" release too if its fine to
> check. YARN-3214 <https://issues.apache.org/jira/browse/YARN-3214> has
> the details for the nodelabel sharing concept.
>
>
>
> Thanks
>
> Sunil
>
>
>
> On Wed, Oct 5, 2016 at 8:14 PM Frank Luo <jluo@merkleinc.com> wrote:
>
> Sunil, thanks for responding.
>
>
>
> So is there any way to dedicate one kind of jobs to certain machines, then
> having those machines be shared if no dedicated job running?
>
>
>
> *From:* Sunil Govind [mailto:sunil.govind@gmail.com]
> *Sent:* Wednesday, October 05, 2016 12:50 AM
> *To:* Frank Luo <jluo@merkleinc.com>; user@hadoop.apache.org;
> user@yarn.apache.org
>
>
> *Subject:* Re: how to add a shareable node label?
>
>
>
> Hi Frank,
>
>
>
> As far as I checked, all labels are "exclusive" in 2.7. In upcoming 2.8
> release, we can get "non-exclusive" or sharable node labels.
>
>
>
> Thanks
>
> Sunil
>
>
>
> On Wed, Oct 5, 2016 at 8:40 AM Frank Luo <jluo@merkleinc.com> wrote:
>
> I am using Hadoop 2.7.3, when I run:
>
> $ yarn rmadmin -addToClusterNodeLabels "Label1(exclusive=false)"
>
>
>
> I got an error as:
>
> … addToClusterNodeLabels: java.io.IOException: label name should only
> contains {0-9, a-z, A-Z, -, _} and should not started with {-,_}
>
>
>
> If I just use “Label1”, it will work fine, but I want a shareable one.
>
>
>
> Anyone knows a better way to do it?
>
> *Access the Q2 2016 Digital Marketing Report for a fresh set of trends and
> benchmarks in digital marketing*
> <http://www2.merkleinc.com/l/47252/2016-07-26/47gt7c>
>
> *Download our latest report titled “The Case for Change: Exploring the
> Myths of Customer-Centric Transformation” *
> <http://www2.merkleinc.com/l/47252/2016-08-04/4b9p7c>
>
> This email and any attachments transmitted with it are intended for use by
> the intended recipient(s) only. If you have received this email in error,
> please notify the sender immediately and then delete it. If you are not the
> intended recipient, you must not keep, use, disclose, copy or distribute
> this email without the author’s prior permission. We take precautions to
> minimize the risk of transmitting software viruses, but we advise you to
> perform your own virus checks on any attachment to this message. We cannot
> accept liability for any loss or damage caused by software viruses. The
> information contained in this communication may be confidential and may be
> subject to the attorney-client privilege.
>

Mime
View raw message