hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Wiley <kwi...@keithwiley.com>
Subject Re: Is perfect control over mapper num AND split distribution possible?
Date Tue, 21 Jan 2014 21:42:45 GMT
Seems to work well.  Thank you very much!

On Jan 21, 2014, at 12:42 , Keith Wiley wrote:

> I'll look it up.  Thanks.
> 
> On Jan 21, 2014, at 11:43 , java8964 wrote:
> 
>> You cannot use hadoop "NLineInputFormat"?
>> 
>> If you generate 100 lines of text file, by default, one line will trigger one mapper
task.
>> 
>> As long as you have 100 task slot available, you will get 100 mapper running concurrently.
>> 
>> You want perfect control over mapper num? NLineInputFormat is designed for your purpose.
>> 
>> Yong


________________________________________________________________________________
Keith Wiley     kwiley@keithwiley.com     keithwiley.com    music.keithwiley.com

"It's a fine line between meticulous and obsessive-compulsive and a slippery
rope between obsessive-compulsive and debilitatingly slow."
                                           --  Keith Wiley
________________________________________________________________________________


Mime
View raw message