hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson" <billy_pear...@sbcglobal.net>
Subject Re: BatchUpdate and BatchOperation
Date Sat, 13 Sep 2008 14:51:22 GMT

What I was doing was merging records for the same row on my map in a 
BatchUpdate then was going to try and merge the BatchUpdates on the reduce 
so all only inserting once per row.

But I have to pass BatchOperations on the map and build the BatchUpdate on 
the Reduce sense we haven no way to get the column/value's of a BatchUpdate 
or merge two BatchUpdates but I can see where it would be helpfull to be 
able to get the column/values from a BatchUpdate.


----- Original Message ----- 
From: "Jim Kellerman" <jim-joqpRgkS6GpWk0Htik3J/w@public.gmane.org>
Newsgroups: gmane.comp.java.hadoop.hbase.user
To: <hbase-user-7ArZoLwFLBtd/SJB6HiN2Ni2O/JbrIOy@public.gmane.org>
Sent: Friday, September 12, 2008 11:09 AM
Subject: RE: BatchUpdate and BatchOperation

BatchUpdate implements Iterable<BatchOperation> doesn't that do what you 

Jim Kellerman, Senior Software Development Engineer
Powerset (Live Search, Microsoft Corporation)

> -----Original Message-----
> From: news [mailto:news-dbVV3NMTNubNLxjTenLetw@public.gmane.org] On Behalf 
> Of Billy Pearson
> Sent: Thursday, September 11, 2008 6:02 PM
> To: hbase-user-7ArZoLwFLBtd/SJB6HiN2Ni2O/JbrIOy@public.gmane.org
> Subject: BatchUpdate and BatchOperation
> should we not have a option to be able to get the BatchOperation's of a
> BatchUpdate
> Example
> I have text files with lines of records I want to insert into hbase
> I was just mapping them and the map was just inserting the records in to
> hbase
> I speed this up by adding inline code to detect multi columns per row that
> are in order.
> I was going to add a reduce stage to merge multi backupdates of the same 
> row
> in the
> reduce stage but there is no way to get the BatchOperations from the
> batchUpdates
> Should we add a method to get the BatchOperations from the BatchUpdates?
> Or Should I just be passing a BatchOperation to the reduce stage then 
> merge
> them into a BatchUpdate?
> Either way I thank we need a method ("add" or something) for BatchUpdate
> that accepts BatchOperations sense this is what we converting to in the 
> end.

View raw message