hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kawa <kawa.a...@gmail.com>
Subject Re: issue about capacity scheduler
Date Thu, 05 Dec 2013 21:53:13 GMT
The heap of application master is controlled
via yarn.app.mapreduce.am.command-opts and its default value is -Xmx1024m (
http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
).

yarn.scheduler.minimum-allocation-mb is completely different property -
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml
The minimum allocation for every container request at the RM, in MBs.
Memory requests lower than this won't take effect, and the specified value
will get allocated at minimum.


2013/12/5 Vinod Kumar Vavilapalli <vinodkv@hortonworks.com>

>
> If both the jobs in the MR queue are from the same user, CapacityScheduler
> will only try to run them one after another. If possible, run them as
> different users. At which point, you will see sharing across jobs because
> they are from different users.
>
> Thanks,
> +Vinod
>
> On Dec 4, 2013, at 1:33 AM, ch huang wrote:
>
> hi,maillist :
>                  i use yarn framework and capacity scheduler  ,and i have
> two queue ,one for hive and the other for big MR job
> in hive  queue it's work fine,because hive task is very faster ,but what i
> think is user A submitted two big MR job ,so first big job eat
> all the resource belongs to the queue ,the other big MR job should wait
> until first job finished ,how can i let the same user 's MR job can run
> parallel?
>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Mime
View raw message