hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kawa <kawa.a...@gmail.com>
Subject Re: issue about capacity scheduler
Date Thu, 05 Dec 2013 21:53:13 GMT
The heap of application master is controlled
via yarn.app.mapreduce.am.command-opts and its default value is -Xmx1024m (

yarn.scheduler.minimum-allocation-mb is completely different property -
The minimum allocation for every container request at the RM, in MBs.
Memory requests lower than this won't take effect, and the specified value
will get allocated at minimum.

2013/12/5 Vinod Kumar Vavilapalli <vinodkv@hortonworks.com>

> If both the jobs in the MR queue are from the same user, CapacityScheduler
> will only try to run them one after another. If possible, run them as
> different users. At which point, you will see sharing across jobs because
> they are from different users.
> Thanks,
> +Vinod
> On Dec 4, 2013, at 1:33 AM, ch huang wrote:
> hi,maillist :
>                  i use yarn framework and capacity scheduler  ,and i have
> two queue ,one for hive and the other for big MR job
> in hive  queue it's work fine,because hive task is very faster ,but what i
> think is user A submitted two big MR job ,so first big job eat
> all the resource belongs to the queue ,the other big MR job should wait
> until first job finished ,how can i let the same user 's MR job can run
> parallel?
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

View raw message