hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-5494) Introduce a zk hosted table-wide read/write lock so only one table operation at a time
Date Thu, 03 May 2012 18:20:55 GMT

     [ https://issues.apache.org/jira/browse/HBASE-5494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HBASE-5494:
-------------------------------

    Attachment: D2997.3.patch

avf requested code review of "[jira] [HBASE-5494] [89-fb] Table-level locks for schema changing
operations.".
Reviewers: Kannan, mbautin, Liyin, JIRA

  Since concurrent modification (e.g., disabling and dropping a table under
  creation) could leave a cluster in an inconsistent state, we need table-level
  locks for schema changing operations.

  A ZooKeeper-based distributed lock has been implemented that
  attempts to create a persistent ZNode (one ZNode per entity being locked, i.e.,
  one per table) if one does not exist. Currently in case a master crashes while
  holding the lock, the lock must be manually removed using the ZooKeeper command
  line (locks being stored in "/hbase/tableLock/").

  The locks implemented are not fair or re-entrant. RecoverableZooKeeper is used
  to correctly handle connection loss.

  To test the locks, InjectionHandler and InjectionEvent have been introduced,
  allowing for injection of arbitrary events, in this case adding delays during
  schema changing operations as to induce a race condition.

  Future work involves automatically deleting stale lock ZNodes upon server
  recovery (providing the attempted operations are not resumed), adding metrics
  around locks (e.g., list all locks held).

TEST PLAN
  Since concurrent modification (e.g., disabling and dropping a table
  under creation) could leave a cluster in an inconsistent state, we
  need table-level locks for schema changing operations.

  A ZooKeeper-based distributed lock has been implemented that attempts
  to create a persistent ZNode (one ZNode per entity being locked, i.e.,
  one per table) if one does not exist. Currently in case a master
  crashes while holding the lock, the lock must be manually removed
  using the ZooKeeper command line (locks being stored in
  "/hbase/tableLock/").

  The locks implemented are not fair or re-entrant. RecoverableZooKeeper
  is used to correctly handle connection loss.

  To test the locks, InjectionHandler and InjectionEvent have been
  introduced, allowing for injection of arbitrary events, in this case
  adding delays during schema changing operations as to induce a race
  condition.

  Future work involves automatically deleting stale lock ZNodes upon
  server recovery (providing the attempted operations are not resumed),
  adding metrics around locks (e.g., list all locks held).


REVISION DETAIL
  https://reviews.facebook.net/D2997

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/HConstants.java
  src/main/java/org/apache/hadoop/hbase/TableLockTimeoutException.java
  src/main/java/org/apache/hadoop/hbase/master/HMaster.java
  src/main/java/org/apache/hadoop/hbase/master/TableLockManager.java
  src/main/java/org/apache/hadoop/hbase/util/InjectionEvent.java
  src/main/java/org/apache/hadoop/hbase/util/InjectionHandler.java
  src/main/java/org/apache/hadoop/hbase/zookeeper/DistributedLock.java
  src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWrapper.java
  src/test/java/org/apache/hadoop/hbase/master/TestSchemaModificationLocks.java
  src/test/java/org/apache/hadoop/hbase/util/DelayInducingInjectionHandler.java
  src/test/java/org/apache/hadoop/hbase/zookeeper/TestDistributedLock.java

                
> Introduce a zk hosted table-wide read/write lock so only one table operation at a time
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-5494
>                 URL: https://issues.apache.org/jira/browse/HBASE-5494
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: stack
>         Attachments: D2997.3.patch
>
>
> I saw this facility over in the accumulo code base.
> Currently we just try to sort out the mess when splits come in during an online schema
edit; somehow we figure we can figure all possible region transition combinations and make
the right call.
> We could try and narrow the number of combinations by taking out a zk table lock when
doing table operations.
> For example, on split or merge, we could take a read-only lock meaning the table can't
be disabled while these are running.
> We could then take a write only lock if we want to ensure the table doesn't change while
disabling or enabling process is happening.
> Shouldn't be too hard to add.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message