hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2713) HA : An alternative approach to clients handling Namenode failover.
Date Wed, 21 Dec 2011 15:33:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174148#comment-13174148
] 

Uma Maheswara Rao G commented on HDFS-2713:
-------------------------------------------

While writing a file if Active Namenode goes down, the thread cannot wait forever until the
Standby Namenode gets fully functional as Active. We place a maximum limit on how long the
user thread can wait for a Namenode to be available. For futher discussions, can term this
limit as namenodeAwaitTimeout. If Namenode is not available even after namenodeAwaitTimeout,
DFS Client will throw an Exception back to the caller. 

Corresponding to Namenode pair, there is a HASwitchAgent which is responsible for managing
connections to multiple Namenode addresses configured. The HASwitchAgent exposes a performFailover()
API, to trigger the switching logic. During any DFS operation, if RetryInvocationHandler finds
that Active Namenode is down, the performFailover() API will be invoked. Then the HASwitchAgent
will execute the switching procedure in a seperate thread, by trying to connect to the two
Namenode addresses configured. This search for finding the Active Namenode will continue until
it successfully connects to an Active Namenode.

The thread which invoked the performFailover() API will go to a TIMED WAIT and will be notified
when the Active Namenode is available. The WAIT will happen for the time configured as 'namenode.await.timeout'.
Once this time has elapsed, an IOException will be thrown from performFailover() API of HASwitchAgent.
The subsequent invocations of the same API may already find that the switching logic is in
progress and just make the calling thread to wait for 'namenode.await.timeout'.
                
> HA : An alternative approach to clients handling  Namenode failover.
> --------------------------------------------------------------------
>
>                 Key: HDFS-2713
>                 URL: https://issues.apache.org/jira/browse/HDFS-2713
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, hdfs client
>    Affects Versions: HA branch (HDFS-1623)
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>
> This is the approach for client failover which we adopted when we developed HA for Hadoop.
I would like to propose thia approach for others to review & include in the HA implementation,
if found useful.
> This is similar to the ConfiguredProxyProvider in the sense that the it takes the address
of both the Namenodes as the input. The major differences I can see from the current implementation
are
> 1) During failover, user threads can be controlled very accurately about *the time they
wait for active namenode* to be available, awaiting the retry. Beyond this, the threads will
not be made to wait; DFS Client will throw an Exception indicating that the operation has
failed.
> 2) Failover happens in a seperate thread, not in the client application threads. The
thread will keep trying to find the Active Namenode until it succeeds. 
> 3) This also means that irrespective of whether the operation's RetryAction is RETRY_FAILOVER
or FAIL, the user thread can trigger the client's failover. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message