hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Tucker (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6788) core-site settings do not propagate correctly when running an MR job.
Date Thu, 29 Sep 2016 17:12:20 GMT
David Tucker created MAPREDUCE-6788:
---------------------------------------

             Summary: core-site settings do not propagate correctly when running an MR job.
                 Key: MAPREDUCE-6788
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6788
             Project: Hadoop Map/Reduce
          Issue Type: Bug
         Environment: HDP 2.3 + kerberos + Ambari 2.1.0
            Reporter: David Tucker


Trying to run a sleep job with DEBUG logs on produces the following error:

15/12/02 17:27:35 DEBUG ssl.FileBasedKeyStoresFactory: CLIENT TrustStore: /etc/security/clientKeys/all.jks
15/12/02 17:27:35 DEBUG impl.TimelineClientImpl: Cannot load customized ssl related configuration.
Fallback to system-generic settings.
java.io.FileNotFoundException: /etc/security/clientKeys/all.jks (No such file or directory)

The workaround that fixed this for me was prepending "`hadoop classpath`:" to mapreduce.application.classpath.

Here is a comparison of the 2:

mapred.application.classpath (before)

$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:
$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:
$PWD/mr-framework/hadoop/share/hadoop/common/*:
$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:
$PWD/mr-framework/hadoop/share/hadoop/yarn/*:
$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:
$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:
$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:
$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:
/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:
/etc/hadoop/conf/secure


`hadoop classpath` (prepended)

/usr/hdp/2.3.2.0-2950/hadoop/conf:
/usr/hdp/2.3.2.0-2950/hadoop/lib/*:
/usr/hdp/2.3.2.0-2950/hadoop/.//*:
/usr/hdp/2.3.2.0-2950/hadoop-hdfs/./:
/usr/hdp/2.3.2.0-2950/hadoop-hdfs/lib/*:
/usr/hdp/2.3.2.0-2950/hadoop-hdfs/.//*:
/usr/hdp/2.3.2.0-2950/hadoop-yarn/lib/*:
/usr/hdp/2.3.2.0-2950/hadoop-yarn/.//*:
/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/lib/*:
/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/.//*:
/usr/share/java/mysql-connector-java-5.1.17.jar:
/usr/share/java/mysql-connector-java.jar:
/usr/hdp/2.3.2.0-2950/tez/*:
/usr/hdp/2.3.2.0-2950/tez/lib/*:
/usr/hdp/2.3.2.0-2950/tez/conf


(Migrated from AMBARI-14205)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message