hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2713) HA : An alternative approach to clients handling Namenode failover.
Date Wed, 21 Dec 2011 21:25:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174419#comment-13174419

Aaron T. Myers commented on HDFS-2713:

bq. Yes , i agree existing implementation also supports.But the main difference is, lets take
a case with existing implementation, after a finite period of time, if it is not able to get
active node proxy instance it will throw exception and fail. After some time if other call
comes, then again it will do failover. But with my proposed one, Background thread will continue
failover indefinitely until it finds active proxy instance. So, by the time next call come
this background thread may make ready of active node proxy.

I still don't see what benefit the background thread has. In the case you describe, with the
current implementation, the second client request (after the failed one which had timed out
retrying/failing over) would just simply succeed, or fail over immediately and then succeed.
So, the background thread won't have saved much if any work, and instead may indefinitely
be doing (potentially unnecessary) work in the background.

What am I missing?
> HA : An alternative approach to clients handling  Namenode failover.
> --------------------------------------------------------------------
>                 Key: HDFS-2713
>                 URL: https://issues.apache.org/jira/browse/HDFS-2713
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, hdfs client
>    Affects Versions: HA branch (HDFS-1623)
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
> This is the approach for client failover which we adopted when we developed HA for Hadoop.
I would like to propose thia approach for others to review & include in the HA implementation,
if found useful.
> This is similar to the ConfiguredProxyProvider in the sense that the it takes the address
of both the Namenodes as the input. The major differences I can see from the current implementation
> 1) During failover, user threads can be controlled very accurately about *the time they
wait for active namenode* to be available, awaiting the retry. Beyond this, the threads will
not be made to wait; DFS Client will throw an Exception indicating that the operation has
> 2) Failover happens in a seperate thread, not in the client application threads. The
thread will keep trying to find the Active Namenode until it succeeds. 
> 3) This also means that irrespective of whether the operation's RetryAction is RETRY_FAILOVER
or FAIL, the user thread can trigger the client's failover. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message