Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of lists@nabble.com designates
 216.139.236.158 as permitted sender)
Message-ID: <20948685.post@talk.nabble.com>
Date: Wed, 10 Dec 2008 19:06:26 -0800 (PST)
From: tigertail <tyczjs@yahoo.com>
To: hbase-user@hadoop.apache.org
Subject: Re: map reduce range of records from hbase table
In-Reply-To: <839ba01c0810082250p6a529a03w7675525d3f9afe8c@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
References: <839ba01c0810080033m73f85aa8sddc736aecf69ed90@mail.gmail.com>
 <48ED2295.3000505@duboce.net>
 <839ba01c0810082250p6a529a03w7675525d3f9afe8c@mail.gmail.com>


Hi Cedric,

Can you share your version of getSplits to feed only a subset of records to
me? I expect your method can select the subset based on row keys as well as
some column values. Thank you.


Cedric Ho wrote:
> 
> Thanks for the solutions, I've tried overriding getSplits and it does
> what I need.
> 
> But for the RowFilter, I guess it would also need to scan through all
> records and do filtering. So wouldn't it be the same if I do the
> filtering myself during the map phrase?
> 
> Cedric
> 
> 
> On Thu, Oct 9, 2008 at 5:13 AM, stack <stack@duboce.net> wrote:
>> Cedric Ho wrote:
>>>
>>> Hi all,
>>>
>>> I am using 0.18.0 and have successfully used data from hbase table as
>>> input to my map/reduce job.
>>>
>>> I wonder how to specify a subset of records from a table instead of
>>> taking all records as input.
>>> Such as a range of the row keys or maybe by specific values of certain
>>> columns.
>>>
>>
>> You'll have to subclass the TableInputFormat.
>>
>> There is an example in the javadoc on subclassing TIF:
>> http://hadoop.apache.org/hbase/docs/r0.18.0/api/org/apache/hadoop/hbase/mapred/TableInputFormatBase.html
>> (Sorry, the example is mangled.  Do a get of the html source to see
>> non-garbled code).
>>
>> The example shows you how to set a filter.  Filters can filter on rows
>> and
>> values.
>>
>> To work against a subset, you'd probably need to play with getSplits  in
>> your subclass.   Default, it  basically eretrns as many splits as there
>> are
>> regions in your table, so its the whole table always.  Filters could stop
>> unwanted rows being returned but maybe its better if the rows weren't
>> considered in the first place; hence the need of getSplits subclassing.
>>
>> St.Ack
>>
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/map-reduce-range-of-records-from-hbase-table-tp19873787p20948685.html
Sent from the HBase User mailing list archive at Nabble.com.