hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Justin Joseph (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2354) Generalize the HAServiceProtocol interface
Date Wed, 26 Oct 2011 14:09:32 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136002#comment-13136002

Justin Joseph commented on HDFS-2354:

The neutral state is not totally a new way of handling network partitions.  For example, consider
the quorum concept supported by Pacemaker to handle split brain scenarios.  


{quote}Whenever quorum is present, pacemaker will go with the majority vote on important decisions.
When quorum is lost (i.e. the cluster is separated into groups where no group has a majority
of votes) the behavior of the pool is determined by the *no-quorum-policy* property:
*	_ignore_ - Do nothing when quorum is lost.
*	_stop_ (default) - stop all resources in the affected cluster partition.
*	_freeze_ - continue running existing resources, but don’t start any stopped ones.
*	_suicide_ - fence all nodes in the affected partition.{quote}

Neutral state is a modification of the stop option in the list above. When quorum is lost,
instead of stopping the Namenode, it is turned into a state where it is neither Active nor
Standby. When quorum is available again, it is turned to either Active or Standby state. The
advantage of keeping the process in neutral state, compared to stopping the process, is about
the time to start servicing the requests after network connection is restored.  The down time
will be very less in case of going to neutral state.

Another approach is to turn the Namenode to Standby state when quorum is lost. In case of
network partition, this may not be of any help since Standby Namenode won’t be able to ping
the Active Namenode & it will just keep retrying. After network is restored, if this Namenode
(which is turned to Standby when quorum is lost) is elected as active again, the time to turn
the Namenode from Standby state to Active state will be higher depending on what is the timeout
configured for RPC calls.

When Zookeeper is used as the distributed coordinator, Neutral state is an effective way of
handling the network partitions. In future, it may be supported by Pacemaker as an option
for *no-quorum-policy*.  

I hope the approach is simple & clear. No one gets elected to Active state or continues
to be in Active state unless a quorum decides so. If quorum is not available, it relinquishes
the role & remains neutral.  Being in neutral state helps to come back to Active state
faster after quorum is available again.

> Generalize the HAServiceProtocol interface
> ------------------------------------------
>                 Key: HDFS-2354
>                 URL: https://issues.apache.org/jira/browse/HDFS-2354
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>            Reporter: Justin Joseph
>            Assignee: Justin Joseph
>         Attachments: HAService_fw_Class_Diagram.JPG, HDFS-2354.1.patch, HDFS-2354.patch,
Namenode HA using Backup Namenode as Hot Standby.pdf
> This JIRA intends to revisit the patches committed for HADOOP-7455 and HDFS-1974 &
to provide more generic interfaces which allows alternative HA implementations to co-exist
complying with HAServiceProtocol.
> Some of the considerations are
> 1) Support life cycle methods (start*() and stop() APIs) in HAServiceProtocol
> 2) Support custom states in HAServiceProtocol
> 3) As per the patch submitted for HDFS-1974, Namenode implements HAService interface.
This needs to be reconsidered.
> I will elaborate on these points, in the form of comments below.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message