hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Rand <stevenjr...@gmail.com>
Subject Can the Backup Node be deployed when dfs.http.policy is HTTPS_ONLY?
Date Tue, 07 Jun 2016 21:57:11 GMT
Hello,

I'm attempting to deploy a Backup Node [1] on a dev cluster where we
specify that all HTTP communication must happen over SSL (dfs.http.policy =
HTTPS_ONLY). The Backup Node fails to start with this exception:

2016-06-07 14:01:01,243 ERROR
org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
java.lang.NullPointerException
at org.apache.hadoop.net.NetUtils.getHostPortString(NetUtils.java:651)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.setRegistration(NameNode.java:617)
at
org.apache.hadoop.hdfs.server.namenode.BackupNode.registerWith(BackupNode.java:366)
at
org.apache.hadoop.hdfs.server.namenode.BackupNode.initialize(BackupNode.java:162)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:838)
at
org.apache.hadoop.hdfs.server.namenode.BackupNode.<init>(BackupNode.java:89)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1519)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1606)

Earlier in the startup logs for the backup node, I see this line:

2016-06-07 14:01:00,736 INFO org.apache.hadoop.hdfs.DFSUtil: Starting
Web-server for hdfs at:https://0.0.0.0:50470

My understanding of this is that instead of running at the URL specified by
dfs.namenode.backup.http-address, the Backup Node's web server is running
at the URL specified by dfs.namenode.https-address. This seems consistent
with the behavior of DFSUtils#httpServerTemplateForNNAndJN. Then because
the web server is not running at the expected HTTP URL, the call to
getHttpAddress() inNameNode#setRegistration finds a null value for the
httpAddress variable in NameNodeHttpServer.

When I set dfs.http.policy to HTTP_AND_HTTPS, the Backup Node happily
starts up with its web server running at the URL specified by
dfs.namenode.backup.http-address. So my question is: Is it possible to
deploy a Backup Node on a cluster where all HTTP communication must happen
over SSL, and if so, how can I fix my configurations? If not, are there
plans to support HTTPS for the Backup Node, and would it make sense for me
to file a ticket (I couldn't find an existing one)?

Thanks,
Steve


[1]
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html#Backup_Node

Mime
View raw message