hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abhishek sharma <absha...@gmail.com>
Subject Re: multiple jobs on the cluster?
Date Tue, 29 Dec 2009 17:30:17 GMT
Hi Mark,

When you submit multiple jobs to the same cluster, these jobs are
queued up at the jobtracker, and executed in FIFO order.

Based on my understanding of the Hadoop FIFO scheduler, the order in
which jobs get executed is determined by two things: (1) priority of
the job. All jobs have the NORMAL priority by default, (2) the start
time of the job.  So in a scenario where all jobs have the same
priority, they will be executed in the order in which they arrive at
the cluster.

If you submit multiple jobs, there is some initial processing that is
done before the job gets executed at the end of which a message
"Running job"+JOBID is printed. At this point, the job has been queued
up at the jobtracker awaiting execution.

Hadoop also comes with other types of scheduler, for example, the Fair
Scheduler (http://hadoop.apache.org/common/docs/current/fair_scheduler.html).

Hope this helps,

On Tue, Dec 29, 2009 at 12:16 PM, Mark Kerzner <markkerzner@gmail.com> wrote:
> Hi,
> what happens when I submit a few jobs on the cluster? To me, it seems like
> they all are running - which I know can't be, because I only have 2 slaves.
> Where do I read about this?
> I am using Cloudera with EC2.
> Thank you,
> Mark

View raw message