Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CAGpTDNcs3a6Ccqo86Py-r3CkKG11EzDwrnOA2MQPDio90tKarA@mail.gmail.com>
References: 
 <CAPQV63VBcHEWmE8bZiF5-vOxmSkXuH+O79f80ceB-qk_9LGqUw@mail.gmail.com>
 <CAGngS9cK3XJ1hnTOG9Pa1RpkbAUwvprTTsL8JQHqw2z4eAZ2OQ@mail.gmail.com>
 <CAPQV63V9Y8x++nRDevOXMeCqmJB2jfSFWZXfmChhWGzAid__JQ@mail.gmail.com>
 <CADcMMgFeGn4A1ymAdHC5eqa-va+=0AbWqtsCxip4Zm3gSwBamA@mail.gmail.com>
 <CAGpTDNcH4TuUrB-7Djbhoc5T0nhZ7sLAcXJbW1EO1nQC0DCGmg@mail.gmail.com>
 <CAPQV63XfDCM=fFYzyPTq7T-B5iOBzz-37f4MDad-Q8EXrYHbiQ@mail.gmail.com>
 <CAGpTDNcs3a6Ccqo86Py-r3CkKG11EzDwrnOA2MQPDio90tKarA@mail.gmail.com>
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
Date: Thu, 11 Oct 2012 16:36:19 -0400
Message-ID: 
 <CAPQV63XXFN7m9PGi1x=9R+SxGj4Ozn7EfOKPL3yGiFGQhS2wpw@mail.gmail.com>
Subject: Re: NoSuchColumnFamilyException with rowcounter
To: user@hbase.apache.org
Content-Type: text/plain; charset=UTF-8

Hi J-D,

I have about 20M rows over 25 regions on 6 nodes. So that mean I
should see something like 6 tasks or even 25, right? And not just 2?
Keys are 128 byte long. Value is 1 byte.

I tried also to update mapreduce.tasktracker.map.tasks.maximum but
this is "the number of map tasks that should be launched on each node,
not the number of nodes to be used for each map task.", so there was
no changes, as expected.

JM

2012/10/11 Jean-Daniel Cryans <jdcryans@apache.org>:
> On Thu, Oct 11, 2012 at 1:20 PM, Jean-Marc Spaggiari
> <jean-marc@spaggiari.org> wrote:
>> I'm now using thsi command line and it's working fine (except for the
>> number of tasks).
>> HADOOP_CLASSPATH=`/home/hbase/hbase-0.94.0/bin/hbase
>> classpath`:`/home/hadoop/hadoop-1.0.3/bin/hadoop classpath`
>> /home/hadoop/hadoop-1.0.3/bin/hadoop jar
>> /home/hbase/hbase-0.94.0/hbase-0.94.1.jar rowcounter
>> -Dhbase.client.scanner.caching=100 -Dmapred.map.tasks=6
>> -Dmapred.map.tasks.speculative.execution=false work_proposed
>>
>> I simply don't know if the -D parameters are taken into consideration
>> since I get the same results (numbers of tasks, time of exec, etc.)
>> with and without them.
>
> Using a higher caching value won't do much good if you don't have a
> lot of rows. Since you didn't include any data like that in your
> email, I won't guess how much 100 would help your case.
>
> The number of map tasks when mapping an HBase table will be the number
> of regions you have in that table. Unfortunately you can't change it
> unless you write your own input format for HBase.
>
> J-D