hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhijie Shen <zs...@hortonworks.com>
Subject Re: Problem when configure the security in hadoop
Date Tue, 07 Jul 2015 21:07:47 GMT
?Not sure about HDFS special setup, but in general, to use HTTPs, you should have your keystore/truststore
generated and config ssl-client.xml and ssl-server.xml properly.


- Zhijie

________________________________
From: Colin Ma <colinmjj@gmail.com>
Sent: Friday, July 03, 2015 1:37 AM
To: user@hadoop.apache.org
Subject: Problem when configure the security in hadoop

Hi,
         I do the security configuration for Hadoop these days, the Kerberos works fine, but
there maybe has some problems on sasl configuration.
         The following is the related configuration in hdfs-site.xml:
    <property>
      <name>dfs.http.policy</name>
      <value>HTTPS_ONLY</value>
    </property>
    <property>
      <name>dfs.data.transfer.protection</name>
      <value>authentication</value>
    </property>

There is no problem to execute the command like:   hdfs dfs -ls /
But when I execute the command:   hdfs dfs -copyToLocal /temp/test.txt .  The following exception
will be thrown:

         015-07-03 14:02:54,715 INFO org.apache.hadoop.hdfs.server.datanode.DataBlockScanner:
Added bpid=BP-271423801-192.168.20.28-1423724265164 to blockPoolScannerMap, new size=1
2015-07-03 14:03:39,963 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: server-511:50010:DataXceiver
error processing unknown operation  src: /192.168.20.28:58422<http://192.168.20.28:58422>
dst: /192.168.20.28:50010<http://192.168.20.28:50010>
java.io.EOFException: Premature EOF: no length prefix available
         at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2203)
         at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessageAndNegotiationCipherOptions(DataTransferSaslUtil.java:233)
         at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.doSaslHandshake(SaslDataTransferServer.java:369)
         at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.getSaslStreams(SaslDataTransferServer.java:297)
         at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.receive(SaslDataTransferServer.java:124)
         at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:183)
         at java.lang.Thread.run(Thread.java:745)
2015-07-03 15:34:39,917 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Sent 1 blockreports
145 blocks total. Took 1 msec to generate and 6 msecs for RPC and NN processing.  Got back
commands org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@1b3bce82<mailto:org.apache.hadoop.hdfs.server.protocol.FinalizeCommand@1b3bce82>

         Just take a look the method doSaslHandshake() of SaslDataTransferClient.java and
SaslDataTransferServer.java, maybe SaslDataTransferClient send a empty response cause this
exception, and I think some mistakes in the configuration caused this problem.
Is there anyone can help to check this problem?
Thanks for your help.

Best regards,

Colin Ma

Mime
View raw message