accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Lynch <patricklync...@aim.com>
Subject Re: Recovering Tables from HDFS
Date Fri, 06 Jul 2012 19:46:38 GMT
Keith and Adam,


Your solution worked, thank you both for your help. Everyone on my end working with Accumulo
wants to say how impressed they are with it overall!


Keep up the good work,
Patrick



-----Original Message-----
From: Keith Turner <keith@deenlo.com>
To: user <user@accumulo.apache.org>
Sent: Thu, Jul 5, 2012 10:30 am
Subject: Re: Recovering Tables from HDFS


Patrick,

In the comments of ACCUMULO-456, I outlined out a procedure for doing
this for 1.4.

By default cloning a table will flush anything in memory.

Keith

On Thu, Jul 5, 2012 at 10:13 AM, Adam Fuchs <afuchs@apache.org> wrote:
> Hi Patrick,
>
> The short answer is yes, but there are a few caveats:
> 1. As you said, information that is sitting in the in-memory map and in the
> write-ahead log will not be in those files. You can periodically call flush
> (Connector.getTableOperations().flush(...)) to guarantee that your data has
> made it into the RFiles.
> 2. Old data that has been deleted may reappear. RFiles can span multiple
> tablets, which happens when tablets split. Often, one of the tablets
> compacts, getting rid of delete keys. However, the file that holds the
> original data is still in HDFS because it is referenced by another tablet
> (or because it has not yet been garbage collected). If you're using Accumulo
> in an append-only fashion, then this will not be a problem.
> 3. For the same reasons as #2, if you're doing any aggregation you might run
> into counts being incorrect.
>
> You might also check out the table cloning feature introduced in 1.4 as a
> means for backing up a table:
> http://accumulo.apache.org/1.4/user_manual/Table_Configuration.html#Cloning_Tables
>
> Cheers,
> Adam
>
>
> On Thu, Jul 5, 2012 at 9:52 AM, <patricklynch33@aim.com> wrote:
>>
>> users@accumulo,
>>
>> I need help understanding if one could recover or backup tables by taking
>> their files stored in HDFS and reattaching them to tablet servers, even
>> though this would mean the loss of information from recent mutations and
>> write ahead logs. The documentation on recovery is focused on the failure of
>> a tablet server, but, in the event of a failure of the master or other
>> situation where the tablet servers cannot be utilized, it would be
>> beneficial to know whether the files in HDFS can be used for recovery.
>>
>> Thanks,
>>
>> Patrick Lynch
>
>

 

Mime
View raw message