hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject [DISCUSS] Clarification on Compatibility Policy: Upgraded Client + Old Server
Date Wed, 19 Mar 2014 20:59:11 GMT
I'd like to discuss clarification of part of our compatibility policy.
 Here is a link to the compatibility documentation for release 2.3.0:

http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common/Compatibility.html#Wire_compatibility

For convenience, here are the specific lines in question:

Client-Server compatibility is required to allow users to continue using
the old clients even after upgrading the server (cluster) to a later
version (or vice versa). For example, a Hadoop 2.1.0 client talking to a
Hadoop 2.3.0 cluster.

Client-Server compatibility is also required to allow upgrading individual
components without upgrading others. For example, upgrade HDFS from version
2.1.0 to 2.2.0 without upgrading MapReduce.

Server-Server compatibility is required to allow mixed versions within an
active cluster so the cluster may be upgraded without downtime in a rolling
fashion.

Notice that there is no specific mention of upgrading the client ahead of
the server.  (There is no clause for "upgraded client + old server".)
 Based on my experience, this is a valid use case when a user wants to pick
up a client-side bug fix ahead of the cluster administrator's upgrade
schedule.

Is it our policy to maintain client compatibility with old clusters within
the same major release?  I think many of us have assumed that the answer is
yes and coded our new features accordingly, but it isn't made explicit in
the documentation.  Do we all agree that the answer is yes, or is it
possibly up for debate depending on the change in question?  In RFC 2119
lingo, is it a MUST or a SHOULD?  Either way, I'd like to update the policy
text to make our decision clear.  After we have consensus, I can volunteer
to file an issue and patch the text of the policy.

This discussion started initially in MAPREDUCE-4052, which involved
changing our scripting syntax for MapReduce YARN container submissions.  We
settled the question there by gating the syntax change behind a
configuration option.  By default, it will continue using the existing
syntax currently understood by the pre-2.4.0 NodeManager, thus preserving
compatibility.  We wanted to open the policy question for wider discussion
though.

Thanks, everyone.

Chris Nauroth
Hortonworks
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message