hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4463) ActiveStandbyElector can join election even before Service HEALTHY, and results in null data at ActiveBreadCrumb
Date Mon, 04 Feb 2013 04:58:13 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinay updated HDFS-4463:
------------------------

    Description: 
ActiveStandbyElector can store null at ActiveBreadCrumb in the below race condition. At further
all failovers will fail resulting NPE.

1. ZKFC restarted.
2. due to machine busy, first zk connection is expired even before the health monitoring returned
the status.
3. On re-establishment transitionToActive will be called, at this time appData will be null,
4. So now ActiveBreadCrumb will have null.
5. After this any failovers will fail throwing 

{noformat}java.lang.NullPointerException
	at org.apache.hadoop.util.StringUtils.byteToHexString(StringUtils.java:171)
	at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:892)
	at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:797)
	at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:475)
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:545)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497){noformat}


Should not join the election before service is HEALTHY


  was:
ActiveStandbyElector can store null at ActiveBreadCrumb in the below race condition. At further
all failovers will fail resulting NPE.

1. ZKFC restarted.
2. due to less machine busy, first zk connection is expired even before the health monitoring
returned the status.
3. On re-establishment transitionToActive will be called, at this time appData will be null,
4. So now ActiveBreadCrumb will have null.
5. After this any failovers will fail throwing 

{noformat}java.lang.NullPointerException
	at org.apache.hadoop.util.StringUtils.byteToHexString(StringUtils.java:171)
	at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:892)
	at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:797)
	at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:475)
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:545)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497){noformat}


Should not join the election before service is HEALTHY


    
> ActiveStandbyElector can join election even before Service HEALTHY, and results in null
data at ActiveBreadCrumb
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-4463
>                 URL: https://issues.apache.org/jira/browse/HDFS-4463
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: ha
>    Affects Versions: 2.0.2-alpha
>            Reporter: Vinay
>            Assignee: Vinay
>            Priority: Critical
>
> ActiveStandbyElector can store null at ActiveBreadCrumb in the below race condition.
At further all failovers will fail resulting NPE.
> 1. ZKFC restarted.
> 2. due to machine busy, first zk connection is expired even before the health monitoring
returned the status.
> 3. On re-establishment transitionToActive will be called, at this time appData will be
null,
> 4. So now ActiveBreadCrumb will have null.
> 5. After this any failovers will fail throwing 
> {noformat}java.lang.NullPointerException
> 	at org.apache.hadoop.util.StringUtils.byteToHexString(StringUtils.java:171)
> 	at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:892)
> 	at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:797)
> 	at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:475)
> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:545)
> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497){noformat}
> Should not join the election before service is HEALTHY

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message