hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12990) Change default NameNode RPC port back to 8020
Date Sat, 13 Jan 2018 17:25:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16325225#comment-16325225

Eric Yang commented on HDFS-12990:

Maybe we should look at NN RPC port as a new feature to ensure the down stream projects really
tested with Hadoop 3 instead of gambling on compatibility.  In Hadoop bylaws, it doesn't mention
compatibility between major versions, only minor version within the same major version.  This
bylaw is challenged by this JIRA because major incompatible change have been made.  This is
greatest challenge to Hadoop policy.  If this policy doesn't hold up when challenged, then
I would suggest to revise the policy first.  Most complains are knee jerk reaction toward
uncertainty toward certifying a new product version.  I don't think this issue is worthy of
revising Hadoop policy.  We should not try to rationalize this is a bug to bend the policy,
otherwise, it would be disrespectful to Hadoop policy, and encourage others to break the rules.

We do not know if down stream projects depend on other ports that were changed.  Therefore,
I would recommend for downstream projects which has hard coded 8020 ports to make new releases
that have properly certified with Hadoop 3.  I found other projects like Chukwa and Ambari
are both immune to NN rpc port change problem.  Code with good design are most likely immune
to this problem.  If we committed this change, future major release might need to break compatibility
again to shake away bad design flaws.  We will be unable to do that for sake of compatibility.
 While I admire Microsoft's ability to enable upgrade from Windows 1.0 to Windows 10, but
Windows still don't preserve all my theme and coloring between Window versions, or make my
old scanner work.  Can Hadoop make the same kind of commitment without making huge investment?
 Having seen multi-generation of programmers retired in IBM.  I know that forever compatibility
is not sustainable and it only cost more in the long run.  We should move on and do something
better for Hadoop.

Please respect my -1 vote because I believe in the wisdom of current version scheme, and port
change ensures the downstream projects really certified with Hadoop 3.

> Change default NameNode RPC port back to 8020
> ---------------------------------------------
>                 Key: HDFS-12990
>                 URL: https://issues.apache.org/jira/browse/HDFS-12990
>             Project: Hadoop HDFS
>          Issue Type: Task
>          Components: namenode
>    Affects Versions: 3.0.0
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
>            Priority: Critical
>         Attachments: HDFS-12990.01.patch
> In HDFS-9427 (HDFS should not default to ephemeral ports), we changed all default ports
to ephemeral ports, which is very appreciated by admin. As part of that change, we also modified
the NN RPC port from the famous 8020 to 9820, to be closer to other ports changed there.
> With more integration going on, it appears that all the other ephemeral port changes
are fine, but the NN RPC port change is painful for downstream on migrating to Hadoop 3. Some
examples include:
> # Hive table locations pointing to hdfs://nn:port/dir
> # Downstream minicluster unit tests that assumed 8020
> # Oozie workflows / downstream scripts that used 8020
> This isn't a problem for HA URLs, since that does not include the port number. But considering
the downstream impact, instead of requiring all of them change their stuff, it would be a
way better experience to leave the NN port unchanged. This will benefit Hadoop 3 adoption
and ease unnecessary upgrade burdens.
> It is of course incompatible, but giving 3.0.0 is just out, IMO it worths to switch the
port back.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message