Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Thu, 11 Dec 2014 03:07:12 +0000 (UTC)
From: "Xuan Gong (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12757721.1416929613000.126311.1418267232671@Atlassian.JIRA>
In-Reply-To: <JIRA.12757721.1416929613000@Atlassian.JIRA>
References: <JIRA.12757721.1416929613000@Atlassian.JIRA>
 <JIRA.12757721.1416929613375@arcas>
Subject: [jira] [Commented] (MAPREDUCE-6173) Document the configuration of
 deploying MR over distributed cache with enabling wired encryption at the
 same time
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242059#comment-14242059 ] 

Xuan Gong commented on MAPREDUCE-6173:
--------------------------------------

The document patch looks fine, but could you rewrite this:
"To fix the problem, create a directory like: sub-directory under $HADOOP_CONF ($HADOOP_HOME/etc/hadoop by default)" ? Maybe put "sub-directory under $HADOOP_CONF ($HADOOP_HOME/etc/hadoop by default)" to the end as an example ?

> Document the configuration of deploying MR over distributed cache with enabling wired encryption at the same time
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6173
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6173
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: distributed-cache, documentation
>    Affects Versions: 2.6.0
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: MAPREDUCE-6173.patch, Screen Shot for MAPREDUCE-6173.png
>
>
> Use the current documented configuration (specified in http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html) to work with the cluster enabling shuffle encryption (http://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html) will cause the job failed with exception below:
> {noformat}
> 2014-10-10 02:17:16,600 WARN [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: Failed to connect to tassapol-centos5nano1-3.cs1cloud.internal:13562 with 1 map outputs
> javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
> 	at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Alerts.java:174)
> 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1731)
> 	at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:241)
> 	at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Handshaker.java:235)
> 	at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1206)
> 	at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:136)
> 	at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Handshaker.java:593)
> 	at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Handshaker.java:529)
> 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:925)
> 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1170)
> 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1197)
> 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1181)
> 	at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:434)
> 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:81)
> 	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.setNewClient(AbstractDelegateHttpsURLConnection.java:61)
> 	at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:584)
> 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1193)
> 	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:379)
> 	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:318)
> 	at org.apache.hadoop.mapreduce.task.reduce.Fetcher.verifyConnection(Fetcher.java:427)
> {noformat}
> This is due to ssl-client.xml is not included in MR tar ball when we deploy it over distributed cache. Putting the ssl-client.xml on CLASSPATH of MR job can resolve the problem and we should document it.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)