hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: Uber Job!
Date Mon, 06 May 2013 14:52:54 GMT
Split creation is primarily InputForma's responsibility, IMHO. It's good if
splits overlap with the block, but it's not always true.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Mon, May 6, 2013 at 8:15 PM, Rahul Bhattacharjee <rahul.rec.dgp@gmail.com
> wrote:

> Hi,
>
> I was going through the definition of Uber Job of Hadoop.
>
> A job is considered uber when it has 10 or less maps , one reducer and the
> complete data is less than one dfs block size.
>
> I have some doubts here-
>
> Splits are created as per the dfs block size.Creating 10 mappers are
> possible from one block of data by some settings change (changing the max
> split size). But trying to understand , why would some job need to run
> around 10 maps for 64 MB of data.
> One thing may be that the job is immensely CUP intensive. Will it be a
> correct assumption? or is there is any other reason for this.
>
> Thanks,
> Rahul
>
>
>

Mime
View raw message