hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1578) First step towards data transter protocol compatibility: support DatanodeProtocol#getDataTransferProtocolVersion
Date Tue, 11 Jan 2011 18:58:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980256#action_12980256
] 

Todd Lipcon commented on HDFS-1578:
-----------------------------------

ah, we are have race conditions commenting here on this JIRA :)

bq. Todd, what's the advantage of making the RPC to NameNode?It looks intuitive to me to query
a datanode to get its data transfer version

I agree, but you said above that you assume that within a cluster, the DN protocol will not
vary - ie that the data transfer version is an attribute of a cluster, not of a particular
node. So, as a cluster-wide attribute, it makes sense to query the master/NN.

Querying just one node and making the assumption all are the same seems a little fragile -
you could easily end up in a state where some of the DNs in the cluster have version A and
others have version B, and clients will be able to talk to some but not others depending on
which one they happen to RPC to first. Will the DNs report their transfer version to the NN
when they register to the cluster, and the NN will reject any DN registration that doesn't
match?

> First step towards data transter protocol compatibility: support DatanodeProtocol#getDataTransferProtocolVersion
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-1578
>                 URL: https://issues.apache.org/jira/browse/HDFS-1578
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.23.0
>
>
> HADOOP-6904 allows us to handle RPC changes in a compatible way. However, we have one
more protocol to take care of, the data transfer protocol, which a dfs client uses to read
data from or write data to a datanode.
>  
> My proposal is to add a new RPC getDataTransferVersion to DatanodeProtocol that returns
the data transfer protocol version running on the datanode. A dfs client gets the datanode's
version number before it reads from/writes to a datanode. With this, the dfs client could
behave differently according to datanode's data transfer version. This provides a base for
us to make data transfer protocol changes in a compatible way.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message