hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6904) A baby step towards inter-version communications between dfs client and NameNode
Date Tue, 30 Nov 2010 19:16:14 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12965354#action_12965354

Sanjay Radia commented on HADOOP-6904:

>major/minor numbers can be considered an optimization, isn't it?
The M-m# allows one to assert compatibility; the other way is via method name comparison.
I see them as complementary rather than
as an optimization. Without the major minor number one has to do a check on each call or do
a check on the getProxy. The real difference is the following:  if you check during getProx()
and a method is missing then how does one  assert that it is still okay to connect  because
 the missing method is actually a new method and the call can return a non-implemented exception;
 with  M-m# one can assert this because when the M# has not changed and hence one is assured
that the method was not deleted and  it must be a new method.

If you look at the example of list() call  I gave above, the method names give a minor simplification
of the code.
I have no issues with passing the method names as long as we also optimize by not sending
the list when there is match of version numbers. I also believe we will have to cache the
results across reconnects (BTW we had discussed the same optimization for 
Avro RPC except that it uses the schema checksum).

> A baby step towards inter-version communications between dfs client and NameNode
> --------------------------------------------------------------------------------
>                 Key: HADOOP-6904
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6904
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: ipc
>    Affects Versions: 0.22.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.22.0
>         Attachments: majorMinorVersion.patch, majorMinorVersion1.patch, rpcVersion.patch,
> Currently RPC communications in Hadoop is very strict. If a client has a different version
from that of the server, a VersionMismatched exception is thrown and the client can not connect
to the server. This force us to update both client and server all at once if a RPC protocol
is changed. But sometime different versions do not mean the client & server are not compatible.
It would be nice if we could relax this restriction and allows us to support inter-version
> My idea is that DfsClient catches VersionMismatched exception when it connects to NameNode.
It then checks if the client & the server is compatible. If yes, it sets the NameNode
version in the dfs client and allows the client to continue talking to NameNode. Otherwise,
rethrow the VersionMismatch exception.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message