hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Garvey <rgarv...@umbc.edu>
Subject Encrypted Shuffle Help
Date Thu, 13 Dec 2012 14:38:19 GMT
Hi,

I am relatively new to Hadoop and completely new to SSL encryption. I am
having issues getting encrypted shuffle working on a small test cluster
with Mapreduce V1. I am using self signed certificates I generated with the
java keytool. I followed the instructions on the site Apache Hadoop website
(
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/EncryptedShuffle.html).
The web interfaces for the nodes are using the correct certificates and the
Map phase of jobs work correctly. However reduce tasks always fail.. I've
been using terasort and teragen as test jobs for the cluster. Terasort
always works and teragen fails on the reduce phase every time. I get the
following error in the logs:

2012-12-12 10:10:47,491 INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201212120947_0001_r_000000_0 0.0% reduce > copy >
2012-12-12 10:10:47,869 WARN org.mortbay.log: EXCEPTION
javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:136)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1806)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:986)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)
	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)
	at org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:708)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)


I generated a key pair for each node in the cluster, Exported each
cert to a trust store and copied the trust store (containing public
keys for every node) to each of the nodes in the cluster. I set both
ssl-server.xml and ssl-client.xml to use the same keystore and
truststore.

I haven't been able to find any documentation on this feature other
than the Apache site and Cloudera's site. Any help would be greatly
appreciated.


Thanks,

Ryan

Mime
View raw message