accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <>
Subject Re: Accumulo / HBase migration
Date Tue, 09 Jul 2013 17:35:31 GMT
I think the easiest/simplest/best thing to do would be to grab the
already sorted data files from HBase, and re-write them to Accumulo's
RFiles (with empty visibility labels or a user-specified one during
the translation), and then simply bulk-import into Accumulo. This
could be done as a map-only mapreduce job.

Christopher L Tubbs II

On Tue, Jul 9, 2013 at 1:26 PM, Donald Miner <> wrote:
> Has anyone developed tools to migrate data from an existing HBase
> implementation to Accumulo? My team has done it "manually" in the past but
> it seems like it would be reasonable to write a process that handled the
> steps in a more automated fashion.
> Here are a few sample designs I've kicked around:
> HBase -> mapreduce -> mappers bulk write to accumulo -> Accumulo
> or
> HBase -> mapreduce -> tfiles via AccumuloFileOutputFormat -> Accumulo bulk
> load -> Accumulo
> or
> HBase -> bulk export -> map-only mapreduce to translate hfiles into tfiles
> (how hard would this be??) -> Accumulo bulk load -> Accumulo
> I guess this could be extended to go the other way around (and also include
> Cassandra perhaps).
> Maybe we'll start working on this soon. I just wanted to kick the idea out
> there to see if it's been done before or if anyone has some gut reactions to
> the process.
> -Don
> This communication is the property of ClearEdge IT Solutions, LLC and may
> contain confidential and/or privileged information. Any review,
> retransmissions, dissemination or other use of or taking of any action in
> reliance upon this information by persons or entities other than the
> intended recipient is prohibited. If you receive this communication in
> error, please immediately notify the sender and destroy all copies of the
> communication and any attachments.

View raw message