accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: Accumulo / HBase migration
Date Tue, 09 Jul 2013 17:35:31 GMT
I think the easiest/simplest/best thing to do would be to grab the
already sorted data files from HBase, and re-write them to Accumulo's
RFiles (with empty visibility labels or a user-specified one during
the translation), and then simply bulk-import into Accumulo. This
could be done as a map-only mapreduce job.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Tue, Jul 9, 2013 at 1:26 PM, Donald Miner <dminer@clearedgeit.com> wrote:
> Has anyone developed tools to migrate data from an existing HBase
> implementation to Accumulo? My team has done it "manually" in the past but
> it seems like it would be reasonable to write a process that handled the
> steps in a more automated fashion.
>
> Here are a few sample designs I've kicked around:
>
> HBase -> mapreduce -> mappers bulk write to accumulo -> Accumulo
> or
> HBase -> mapreduce -> tfiles via AccumuloFileOutputFormat -> Accumulo bulk
> load -> Accumulo
> or
> HBase -> bulk export -> map-only mapreduce to translate hfiles into tfiles
> (how hard would this be??) -> Accumulo bulk load -> Accumulo
>
> I guess this could be extended to go the other way around (and also include
> Cassandra perhaps).
>
> Maybe we'll start working on this soon. I just wanted to kick the idea out
> there to see if it's been done before or if anyone has some gut reactions to
> the process.
>
> -Don
>
> This communication is the property of ClearEdge IT Solutions, LLC and may
> contain confidential and/or privileged information. Any review,
> retransmissions, dissemination or other use of or taking of any action in
> reliance upon this information by persons or entities other than the
> intended recipient is prohibited. If you receive this communication in
> error, please immediately notify the sender and destroy all copies of the
> communication and any attachments.

Mime
View raw message