hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish Kumar Singh <ashish23...@gmail.com>
Subject Application Master waits a long time after Mapper/Reducers finish
Date Mon, 20 Jul 2015 08:45:35 GMT
Hello Users ,

I am facing a problem running Mapreduce jobs on Hadoop 2.6.
I am observing that the Applocation Master  waits for a long time after all
the Mappers and Reducers are completed before the job is completed .

This wait time sometimes exceeds 20-25 mins which is very strange as our
mappers and reducers complete in less than 10 minutes for the job .

Below are some observations:
a) Job completion status stands at 95% when the wait begins

b)JOB_COMMIT is initiated just before this wait time ( logs: 2015-07-14
01:54:46,636 INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
job_1436854849540_0123Job Transitioned from RUNNING to COMMITTING )

c) job success happens after 20-25 minutes ( logs: 2015-07-14 02:15:06,634
INFO [AsyncDispatcher event handler]
org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
job_1436854849540_0123Job Transitioned from COMMITTING to SUCCEEDED )


Appreciate any help on this .

Thread dump while the Application master hangs is attached.

Regards,
Ashish

Mime
View raw message