hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Moore" <jamesthepi...@gmail.com>
Subject Re: Determining number of mappers and number of input splits
Date Fri, 01 Aug 2008 22:14:56 GMT
On Wed, Jul 30, 2008 at 11:24 PM, Naama Kraus <naamakraus@gmail.com> wrote:
> Hi,
>
> I am a bit confused of how the framework determines the number of mappers of
> a job and the number of input splits.
> Could anyone summarize ?

Take a look at http://wiki.apache.org/hadoop/HowManyMapsAndReduces

Things start to become a little more clear when you think about
Hadoop-size datasets.  It's common that you usually care about tuning
the number of simultaneous jobs running on a single machine (one per
core?  one per hard drive? one per <whatever>?), and the total number
is just "many."

-- 
James Moore | james@restphone.com
Ruby and Ruby on Rails consulting
blog.restphone.com

Mime
View raw message