Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: unknown (athena.apache.org: error in processing during lookup of
 mingma@ebay.com)
DomainKey-Signature: s=corp; d=ebay.com; c=nofws; q=dns;
  h=X-EBay-Corp:X-IronPort-AV:Received:Received:From:To:Date:
   Subject:Thread-Topic:Thread-Index:Message-ID:References:
   In-Reply-To:Accept-Language:Content-Language:
   X-MS-Has-Attach:X-MS-TNEF-Correlator:acceptlanguage:
   x-ems-proccessed:x-ems-stamp:Content-Type:
   Content-Transfer-Encoding:MIME-Version:X-CFilter;
  b=SAKhSgtvDXf5Fmf7lsttoG6WKrTvbHVxb7XInTppO84s2tfU1dScH/dI
   nHqt05gX56yowD/ZNt+WaJ+Ft8YiUGzQ+oBrLi94ehIOO2/TO+sorcsqR
   e19DO7rjtNuSUJF;
From: "Ma, Ming" <mingma@ebay.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Date: Sun, 28 Aug 2011 22:58:46 -0600
Subject: RE: Number of map jobs per region
Thread-Topic: Number of map jobs per region
Thread-Index: AcxlYfZTxQsH+EB1Q3Scz+zmiSoZ6QApMf4Q
Message-ID: 
 <D10C58C9BA49DC40AA68056049009BF3646452B25B@DEN-MEXMS-001.corp.ebay.com>
References: 
 <CAKo4JerJ7AscgmU2cv89-gCB_Mh7hbpKJawT+zJV4X_OyDC5Pw@mail.gmail.com>
In-Reply-To: 
 <CAKo4JerJ7AscgmU2cv89-gCB_Mh7hbpKJawT+zJV4X_OyDC5Pw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Dhaval,

You might find https://issues.apache.org/jira/browse/HBASE-4063 useful when=
 it is ready. Of course, you can always use your own customized version of =
TableInputFormat. https://issues.apache.org/jira/browse/HBASE-4039 allows y=
ou to provide your own TableInputFormat to TableMapReduceUtil.

Ming

-----Original Message-----
From: Dhaval Makawana [mailto:dhaval.makawana@gmail.com]=20
Sent: Sunday, August 28, 2011 2:06 AM
To: user@hbase.apache.org
Subject: Number of map jobs per region

Hi,

We have 31 regions for a table in our HBase system and hence while scanning
the table via TableMapper, it creates 31 maps. Following is the line from
documentation where I got the reason for the same.

"Reading from HBase, the TableInputFormat asks HBase for the list of region=
s
and makes a map-per-region or mapred.map.tasks maps, whichever is smaller "
(
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/package-s=
ummary.html
)

Each region file size is almost 7 GB(lzo compressed  data) and map jobs are
taking huge time to processed the data. Is there any way to increase
parallelism(allocate more maps per region)?

Regards,
Dhaval