hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2713) HA : An alternative approach to clients handling Namenode failover.
Date Wed, 21 Dec 2011 19:37:31 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174347#comment-13174347
] 

Aaron T. Myers commented on HDFS-2713:
--------------------------------------

Hey Uma, there are several things I don't understand about this proposal.

bq. 1) During failover, user threads can be controlled very accurately about the time they
wait for active namenode to be available, awaiting the retry. Beyond this, the threads will
not be made to wait; DFS Client will throw an Exception indicating that the operation has
failed.

The current system already supports this. Clients will failover and retry with some random,
exponential backoff for a finite period of time, after which the operation will fail, throwing
an exception.

bq. 2) Failover happens in a seperate thread, not in the client application threads. The thread
will keep trying to find the Active Namenode until it succeeds. 

What's the point of doing this in a separate thread? Given that client operations still block
while the failover is attempted, it doesn't seem like this difference will be tangible to
the user.

3) This also means that irrespective of whether the operation's RetryAction is RETRY_FAILOVER
or FAIL, the user thread can trigger the client's failover.

This confuses me. How does this work?

In short, this proposal just seems _different_ and not necessarily _better_ than the current
implementation. This implementation also seems like a more complex design to me, so without
tangible user benefits I don't see much point in doing it.

The other thing that's not clear to me is how you'd propose to incorporate it into HDFS. Would
it be an alternative to the current implementation? Or done as an enhancement to the current
implementation?
                
> HA : An alternative approach to clients handling  Namenode failover.
> --------------------------------------------------------------------
>
>                 Key: HDFS-2713
>                 URL: https://issues.apache.org/jira/browse/HDFS-2713
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, hdfs client
>    Affects Versions: HA branch (HDFS-1623)
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>
> This is the approach for client failover which we adopted when we developed HA for Hadoop.
I would like to propose thia approach for others to review & include in the HA implementation,
if found useful.
> This is similar to the ConfiguredProxyProvider in the sense that the it takes the address
of both the Namenodes as the input. The major differences I can see from the current implementation
are
> 1) During failover, user threads can be controlled very accurately about *the time they
wait for active namenode* to be available, awaiting the retry. Beyond this, the threads will
not be made to wait; DFS Client will throw an Exception indicating that the operation has
failed.
> 2) Failover happens in a seperate thread, not in the client application threads. The
thread will keep trying to find the Active Namenode until it succeeds. 
> 3) This also means that irrespective of whether the operation's RetryAction is RETRY_FAILOVER
or FAIL, the user thread can trigger the client's failover. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message