hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ninad Raut <hbase.user.ni...@gmail.com>
Subject Re: Why only few map tasks are running at a time inspite of plenty of scope for remaining?
Date Fri, 24 Jul 2009 10:30:34 GMT
If your data is stored just on one regionserver you will have only one map
inspite of setting
 conf.set("mapred.tasktracker.map.tasks.maximum", "2");
there are two approaches:

On Fri, Jul 24, 2009 at 4:54 AM, akhil1988 <akhilanger@gmail.com> wrote:

> Hi all,
> I am using a HTable as input to my map jobs and my reducer outputs to
> another Htable. There are 10 regions of my input HTable. And I have set
>        conf.set("mapred.tasktracker.map.tasks.maximum", "2");
>        conf.set("mapred.tasktracker.map.tasks.maximum", "2");
>       c.setNumReduceTasks(26);
> My cluster contains 15 nodes(out of which 2 are maters). When I run my job,
> only 2 map tasks run at a time and the remaining 8 are shown as pending. 24
> reduce tasks(out of 26) also get started initially and remaing 2 are shown
> as pending. I am confused why only 2 tasks are running at a time, though
> there are a total of 26 slots for map tasks.
> However, this does not happen when I run jobs in which I take files as
> inputs(i.e. only simple MapReduce jobs and not involving HBase at all).
> Only
> when a Htable is taken as input very few map tasks run concurrently than
> expected.
> Can anyone suggest why this is happening?
> What I have observed in simple mapreduce jobs that first all map tasks are
> instantiated and then reduce tasks. But  this does not seem to be happening
> in HTable case??
> --
> View this message in context:
> http://www.nabble.com/Why-only-few-map-tasks-are-running-at-a-time-inspite-of-plenty-of-scope-for-remaining--tp24636315p24636315.html
> Sent from the HBase User mailing list archive at Nabble.com.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message