hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaowei Zhu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12103) libhdfs++: Provide workaround to support cancel on filesystem connect until HDFS-11437 is resolved
Date Fri, 07 Jul 2017 22:08:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078771#comment-16078771
] 

Xiaowei Zhu commented on HDFS-12103:
------------------------------------

+1. The workaround looks reasonable to me.

> libhdfs++: Provide workaround to support cancel on filesystem connect until HDFS-11437
is resolved
> --------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-12103
>                 URL: https://issues.apache.org/jira/browse/HDFS-12103
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: James Clampffer
>            Assignee: James Clampffer
>         Attachments: HDFS-12103.HDFS-8707.000.patch
>
>
> HDFS-11437 is going to take a non-trivial amount of work to do right.  In the meantime
it'd be nice to have a way to cancel pending connections (even when the FS claimed they are
finished).  
> Proposed workaround is to relax the rules about when FileSystem::CancelPending connect
can be called since it isn't able to properly determine when it's connected anyway.  In order
to determine when the FS has connected you can do some simple RPC call since that will wait
on failover.  If CancelPending can be called during that first RPC call then it will effectively
be canceling FileSystem::Connect
> Current cancel rules - asterisk on steps where CancelPending is allowed
> FileSystem::Connect called
> FileSystem communicates with first NN *
> FileSystem::Connect returns - even if it hasn't communicated with the active NN
> Proposed relaxation
> FileSystem::Connect called
> FileSystem communicates with first NN*
> FileSystem::Connect returns *
> FileSystem::GetFileInfo called * -any namenode RPC call will do, ignore perm errors
> RPC engine blocks until it hits the active or runs out of retries *
> FileSystem::GetFileInfo returns
> It'd be up to the user to add in the dummy NN RPC call.  Once HDFS-11437 is fixed this
workaround can be removed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message