accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krishmin Rai <kr...@missionfoc.us>
Subject Re: Re-init Accumulo over existing installation
Date Thu, 05 Jul 2012 18:06:14 GMT
Keith, I created Accumulo-671 about a ZK re-initialization process.

John: definitely not using /tmp as the ZK data dir, but if no one else has experienced this
kind of data loss with ZK, I'm ready to chalk it up to some kind of one-time user error on
our part. If it happens again, I'll dig in further.

Thanks for the quick responses!
Krishmin

On Jul 5, 2012, at 11:35 AM, Keith Turner wrote:

> I have never seen this happen, I have found zookeeper to be very
> reliable.  I think Accumulo needs a utility to handle this case of
> reinitializing just zookeeper.  Would you like to open a ticket?
> 
> Zookeeper does store some important persistent info, like mappings of
> table names to table ids, table config, users data, and FATE ops.
> 
> 
> On Thu, Jul 5, 2012 at 11:15 AM, Krishmin Rai <krrai@missionfoc.us> wrote:
>> Hi All,
>>  We've recently encountered a strange situation on a small test cluster: after an
awkward crash, our ZooKeeper data was erased and we no longer have the [accumulo] znode. The
HDFS accumulo directory is intact, so all the RFiles and etc are still there, but it's not
clear how best to bring Accumulo back up to its previous state. Obviously just starting Accumulo
as-is complains about the missing znode ("Waiting for accumulo to be initialized"), whereas
re-initializing is not possible over existing HDFS directories ("It appears this location
was previously initialized, exiting").
>> 
>>  A couple of questions about recovery strategies:
>> 
>> 1) Is there any way to re-create the znode for a previous instance-id? My understanding
is that ZK is mostly used to store ephemeral data (such as which tserver is currently responsible
for which tablets) and things like users (which we could re-create), so perhaps this is plausible?
>> 
>> 2) I imagine that I could init Accumulo with a new instance.dfs.dir, then import
the RFiles from the old installation back in. I see Patrick just asked a related question,
so, with the data integrity caveats, I would essentially be following the last of the steps
in ACCUMULO-456.
>> 
>> 3) This is a vague question, but have any of you had experience with the [accumulo]
znode being entirely deleted? Aside from stopping/starting ZK (3.3.5) and Accumulo 1.4.0 (possibly
with a force-quit), I'm not sure what we could have done to actually delete it.
>> 
>> This is just a test instance, and the data could easily be recreated, but I want
to take this opportunity to learn a little more about Accumulo plumbing and maintenance.
>> 
>> Thanks,
>> Krishmin
>> 
>> 
>> 
>> 


Mime
View raw message