hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Asif Jan <Asif....@unige.ch>
Subject Re: Single Node with multiple mappers?
Date Fri, 16 Jul 2010 10:03:07 GMT
how is your data being spilt ?
  using   mapred.map.tasks property should let you specify how many  
maps you would want to run (provided your input file is big enough to  
be spilt into multiple chunks)


On Jul 16, 2010, at 11:03 AM, Moritz Krog wrote:

> Hi everyone,
> I was curious if there is any option to use Hadoop in single node mode
> in a way, that enables the process to use more system ressources.
> Right now, Hadoop uses one mapper and one reducer, leaving my i7 with
> about 20% CPU usage (1 core for Hadoop, .5 cores for my OS) basically
> idling.
> Raising the number of map tasks doesn't seem to do much, as this
> parameter seems to more of a hint anyway. Still, I have lots of CPU
> time and RAM left. Any hints on how to use them?
> thanks in advance,
> Moritz

View raw message