Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.214.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <503E1BEB.7060003@bnl.gov>
References: <FD7035C6-E02E-4C90-9890-16DC179C31FD@yahoo.com>
 <D59B4EF5-3C01-418E-B940-302964B9C2C8@yahoo.com>
 <CACvfG=p2ujxXZgaO18FE6G84pmoPD5kkx-LEsPJ_tff7f1jNiw@mail.gmail.com>
 <53903ED9-146F-40FF-BD3B-183E1A95A3AB@yahoo.com> <503E1BEB.7060003@bnl.gov>
From: Harsh J <harsh@cloudera.com>
Date: Wed, 29 Aug 2012 21:31:26 +0530
Message-ID: 
 <CAOcnVr0akUUnnTmn_ncn6dSNo3jfJAGVhzc-U1JAdv93RD51CA@mail.gmail.com>
Subject: Re: Delays in worker node jobs
To: user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hey Terry,

Can you look at your JobTracker logs, grep it for this worker node's
hostname and see the task assignment timestamps vs. when the task
began in real (from the TaskTracker log, grepping for the same attempt
ID)?

On Wed, Aug 29, 2012 at 7:10 PM, Terry Healy <thealy@bnl.gov> wrote:
> Running 1.0.2, in this case on Linux.
>
> I was watching the processes / loads on one TaskTracker instance and
> noticed that it completed it's first 8 map tasks and reported 8 free
> slots (the max for this system). It then waited doing nothing for more
> than 30 seconds before the next "batch" of work came in and started running.
>
> Likewise it also has relatively long periods with all 8 cores running at
> or near idle. There are no jobs failing or obvious errors in the
> TaskTracker log.
>
> What could be causing this?
>
> Should I increase the number of map jobs to greater than number of cores
> to try and keep it busier?
>
> -Terry


-- 
Harsh J