hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phani <phan...@ovi.com>
Subject Re: issue with map running time
Date Sat, 07 Jul 2012 07:24:23 GMT
Other users might have consumed all map slots which may have caused long wait times for some
mapper in your job. In such cases I would watch the queues closely and reconsider job distribution
to grid queues with sufficient map slots.

Thanks,
Phani

 
Best Regards, Phani phani83@ovi.com


>________________________________
> From: Robert Evans <evans@yahoo-inc.com>
>To: "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org> 
>Sent: Friday, 6 July 2012 10:30 PM
>Subject: Re: issue with map running time
> 
>
>How long a program takes to run depends on a lot of things.  It could be a connectivity
issue, or it could be that your program does a lot more processing for some input records
then for others, or it could be that some of your records are a lot smaller so that more of
them exist in a single input split.  Without knowing what the code is doing it is hard to
say more then that.
>
>
>--Bobby Evans 
>
>From:  Kasi Subrahmanyam <kasisubbu440@gmail.com>
>Reply-To:  "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
>To:  "mapreduce-user@hadoop.apache.org" <mapreduce-user@hadoop.apache.org>
>Subject:  issue with map running time
>
>
>
>Hi ,
>
>I have a job which has let us say 10 mappers running in parallel.
>Some are running fast but few of them are taking too long to run.
>For example few mappers are taking 5 to 10 mins but others are taking around 12 hours
or more.
>Does the difference in the data handled by the mappers can cause such a variation or is
it the issue with connectivity.
>
>Note:The cluster we are using have multiple users running their jobs on it.
>
>Thanks in advance.
>Subbu
>
>
>


---
Sent via Epic Browser
Mime
View raw message