hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <dhvanika.s...@wipro.com>
Subject RE: Changing default scheduler in hadoop
Date Sun, 13 Apr 2014 17:27:32 GMT
Hi Hadoop Users,

I am wondering if it is possible to use any other schedulers like SLURM/PBS Pro as default
schedulers to scheduler MR Jobs

Regards
Dhvanika

-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com] 
Sent: Sunday, April 13, 2014 10:54 PM
To: <user@hadoop.apache.org>
Cc: ektacloudstack@gmail.com; common-user@hadoop.apache.org; hdfs-user@hadoop.apache.org
Subject: Re: Changing default scheduler in hadoop

Hi,

On Sun, Apr 13, 2014 at 10:47 AM, Mahesh Khandewal <mahesh.k.exp@gmail.com> wrote:
> Sir i am using Hadoop 1.1.2
> I don't know where is the code residing of default scheduler?

Doing a simple 'find' in the source checkout for name pattern 'Scheduler' should reveal pretty
relevant hits. We do name our Java classes seriously :)

https://github.com/apache/hadoop-common/blob/release-1.1.2/src/mapred/org/apache/hadoop/mapred/JobQueueTaskScheduler.java

> I want to change the default scheduler to fair how can i do this??

You can override the mapred-site.xml placed property 'mapred.jobtracker.taskScheduler' to
specify a custom scheduler (or a supplied one, such as Fair [http://hadoop.apache.org/docs/r1.1.2/fair_scheduler.html]
or Capacity [http://hadoop.apache.org/docs/r1.1.2/capacity_scheduler.html]
Schedulers).

> And if i want to get back to default scheduler how can i do this?

Remove the configuration override, and it will always go back to the default FIFO based scheduler,
the same whose source has been linked above.

> I am struggling since 4 months to get help on Apache Hadoop??

Are you unsure about this?

--
Harsh J

The information contained in this electronic message and any attachments to this message are
intended for the exclusive use of the addressee(s) and may contain proprietary, confidential
or privileged information. If you are not the intended recipient, you should not disseminate,
distribute or copy this e-mail. Please notify the sender immediately and destroy all copies
of this message and any attachments. 

WARNING: Computer viruses can be transmitted via email. The recipient should check this email
and any attachments for the presence of viruses. The company accepts no liability for any
damage caused by any virus transmitted by this email.

www.wipro.com

Mime
View raw message