hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ricky l <rickylee0...@gmail.com>
Subject Re: Capacity Scheduler capacity vs. maximum-capacity
Date Mon, 24 Feb 2014 23:28:33 GMT
Thank Arun for the update. That's very helpful - I was going to switch
to fair scheduler for the preemption purpose.

On Mon, Feb 24, 2014 at 5:51 PM, Arun C Murthy <acm@hortonworks.com> wrote:
> Yes, it's a recent addition to CapacityScheduler.
>
> It's available in hadoop-2.2.0 onwards. See
> https://issues.apache.org/jira/browse/YARN-569.
>
> Arun
>
> On Feb 21, 2014, at 8:28 AM, ricky l <rickylee0815@gmail.com> wrote:
>
> Does Hadoop capacity scheduler support preemption in this scenario?
> Based on what Vinod says, the preemption seems to be supported by
> configuration. If so, can someone point me an instruction to do that?
> The preemption would really be helpful for my use-case. thanks.
>
> On Fri, Feb 21, 2014 at 12:39 AM, Vinod Kumar Vavilapalli
> <vinodkv@apache.org> wrote:
>
>
> Yes, it does take those extra resources away back to queue B. How quickly it
> takes them away depends on whether preemption is enabled or not. If
> preemption is not enabled, it 'takes away' as and when containers from queue
> A start finishing.
>
> +Binod
>
> On Feb 19, 2014, at 5:35 PM, Alex Nastetsky <anastetsky@spryinc.com> wrote:
>
> Will the scheduler take away the 10% from queue B and give it back to queue
> A even if queue B needs it? If not, it would seem that the scheduler is
> reneging on its guarantee.
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader of
> this message is not the intended recipient, you are hereby notified that any
> printing, copying, dissemination, distribution, disclosure or forwarding of
> this communication is strictly prohibited. If you have received this
> communication in error, please contact the sender immediately and delete it
> from your system. Thank You.
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader of
> this message is not the intended recipient, you are hereby notified that any
> printing, copying, dissemination, distribution, disclosure or forwarding of
> this communication is strictly prohibited. If you have received this
> communication in error, please contact the sender immediately and delete it
> from your system. Thank You.

Mime
View raw message