hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhe Li <allenlee...@gmail.com>
Subject Re: Is there any way to limit the concurrent running mappers per job?
Date Tue, 21 Apr 2015 07:23:05 GMT
mapreduce.jobtracker.taskscheduler.maxrunningtasks.perjob can not works
either.

Forgot to mention, I'm using hadoop-2.3.0-cdh5.1.0.
I greped the source code, mapred.jobtracker.taskScheduler.maxRunningTasksPerJob
is only available in
src/hadoop-mapreduce1-project/src/mapred/org/apache/hadoop/mapred/LimitTasksPerJobTaskScheduler.java.
I think it's compatible with YARN's scheculer.

On Tue, Apr 21, 2015 at 2:24 PM, Sanjeev Tripurari <
sanjeev.tripurari@inmobi.com> wrote:

> Hi,
>
> Check if this works for you,
> mapreduce.jobtracker.taskscheduler.maxrunningtasks.perjob
>
> Some properties have been changed with yarn implementation
>
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
>
> -Sanjeev
>
>
>
> On Tue, Apr 21, 2015 at 4:32 AM, Zhe Li <allenlee.lz@gmail.com> wrote:
>
>> Hi, after upgraded to Hadoop 2 (yarn), I found that
>> 'mapred.jobtracker.taskScheduler.maxRunningTasksPerJob' no longer worked,
>> right?
>>
>> One workaround is to use queue to limit it, but it's not easy to control
>> it from job submitter.
>> Is there any way to limit the concurrent running mappers per job?
>> Any documents or discussions before?
>>
>> BTW, any way to search this mailing list before I post a new question?
>>
>> Thanks very much.
>>
>
>
> _____________________________________________________________
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.

Mime
View raw message