hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From llpind <sonny_h...@hotmail.com>
Subject Re: Help with Map/Reduce program
Date Wed, 10 Jun 2009 01:46:24 GMT

Yeah I noticed that shortly after I posted.  I have it "colFam1:" now.    I'm
positive my table has that column family, but my output table still has
nothing in it.

I'm looking at the source code for rowcounter, and it doesn't even require
:.  does it need to be passed in?


I may be going about this wrong, I'm open to ideas.  I need a way to iterate
over an entire HBase table, and count columns (column counter instead of
rowcounter). 

Billy Pearson-2 wrote:
> 
> try with out the * in the column
> "colFam1:*",
> try
> "colFam1:",
> 
> I do not thank the * works like a all option just leave it blank colFam1: 
> and it will give all results
> 
> Billy
> 
> 
> "llpind" <sonny_heer@hotmail.com> wrote in 
> message news:23952252.post@talk.nabble.com...
>>
>> Hi again,
>>
>> I need some help with a map/reduce program I have which copies data from 
>> one
>> table to another.  What I would like to do is iterate through an entire
>> HBase table, and for a given row key and column family count the number
>> of
>> records.  So the output table will have a single column family named 
>> 'count'
>> (e.g. entry would look something like 'rowKey1', 
>> 'count:count_for_rowkey1',
>> '534', where the rowkey could be the same as input table ).
>>
>> here is my first attempt:
>>
>> CONF:
>> =========================================================================
>>
>> c.setInputFormat(TableInputFormat.class);
>> c.setOutputFormat(TableOutputFormat.class);
>>
>>     TableMapReduceUtil.initTableMapJob("inputTableName", "colFam1:*",
>> MapperClass.class,
>>           ImmutableBytesWritable.class, RowResult.class, c);
>>
>>     TableMapReduceUtil.initTableReduceJob("outputTableName",
>> ReducerClass.class, c );
>>
>> MapperClass:
>> =====================================================================
>>
>> @Override
>> public void map(
>> ImmutableBytesWritable key,
>> RowResult row,
>> OutputCollector<ImmutableBytesWritable, RowResult> collector,
>> Reporter reporter) throws IOException {
>>
>>
>> reporter.incrCounter(Counters.ROWS, 1);
>> collector.collect(key, row);
>> }
>>
>> ReducerClass:================================================================
>>
>> @Override
>> public void reduce(ImmutableBytesWritable k,
>> Iterator<RowResult> v,
>> OutputCollector<ImmutableBytesWritable, BatchUpdate> c,
>> Reporter r) throws IOException {
>>
>> while (v.hasNext()){
>> BatchUpdate bu = new BatchUpdate(k.get());
>> while (v.hasNext()){
>> RowResult row = v.next();
>> bu.put(Bytes.toBytes("count:rowToCountName"),
>> Bytes.toBytes(row.size()));
>> }
>> c.collect(k, bu);
>> }
>>                             }
>>
>> ========================================================================
>>
>> It runs the map/reduce, but I get nothing in my output table.
>>
>> Thanks.
>>
>> llpind
>> -- 
>> View this message in context: 
>> http://www.nabble.com/Help-with-Map-Reduce-program-tp23952252p23952252.html
>> Sent from the HBase User mailing list archive at Nabble.com.
>>
>> 
> 
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Help-with-Map-Reduce-program-tp23952252p23954242.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message