accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-1771) Instance id and version info only stored on one hdfs instance
Date Wed, 06 Nov 2013 17:55:20 GMT


Keith Turner commented on ACCUMULO-1771:

bq. Is this a dupe or abstraction of ACCUMULO-1766?

Its not a dupe of ACCUMULO-1766, but its caused by different code making the same assumption.
 The code treats the first configured URI specially.   I am thinking the list of uris should
be used as list of possible places to store new files.   The configured list of uris does
not represent all of the locations that Accumulo has stored files in the past.  

 # User initializes with URI_1 in config, runs w/ this for a bit, then changes config to contain
URI_2 and URI_3.
 # User initializes with URI_1 and URI_2, runs w/ this for a bit, then changes config to URI_0,
URI_1, and URI_2
 # User initializes with URI_1 and URI_2, user wants to reinit so they clear URI_1 (but forget
to clear URI_2), User initializes again with URI_1 and URI_2
 # User initializes instance 1 with URI_1.  User initializes instance 2 with URI_2.   User
shuts down both instances and adds URI_2 to instance 1 config.

In scenario 1 Accumulo will probablly not start because because it will look for the instance
id and data version at URI_2.  In secnario 2 Accumulo will look for instance id and data version
at URI_0.   In scenario 3 initialization may succeed even though URI_2 is dirty.  Scenario
4 shows adding a dirty URI to an existing instance.

A possible solution is the following.
 * init checks and initializes all URIs configured at the time it runs (upgrade would also
need to do this)
 * master, tserver, etc check the set of URIs when they start and ensure instance id and data
version are consistent across all.  If server processes ever reload this prop, the check would
need to be redone.
 * provide a utility to initialize a new URI.  This utility would fail if the URI is not clean.
 The utility would have to be run before adding the URI to the Accumulo config. 

Another possible solution is to store the list of URIs in zookeeper and provide instance operations
for mutating that list.

> Instance id and version info only stored on one hdfs instance
> -------------------------------------------------------------
>                 Key: ACCUMULO-1771
>                 URL:
>             Project: Accumulo
>          Issue Type: Sub-task
>          Components: master, tserver
>            Reporter: Keith Turner
>            Priority: Critical
>             Fix For: 1.6.0
> When instance.volumes contains multiple uris only the first one is initialized with the
data version and instance id.   If the user changes the order of the uris in the config, it
seems like Accumulo will stop working correctly.

This message was sent by Atlassian JIRA

View raw message