ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-14764) MapReduce, Oozie service check failures after upgrade finished
Date Fri, 05 Feb 2016 20:57:39 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-14764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15134973#comment-15134973
] 

Hudson commented on AMBARI-14764:
---------------------------------

SUCCESS: Integrated in Ambari-branch-2.2 #298 (See [https://builds.apache.org/job/Ambari-branch-2.2/298/])
AMBARI-14764. MapReduce, Oozie service check failures after upgrade (dlysnichenko: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d2b3a805414346646ec1dd1665a1e2a4d1d653e6])
* ambari-server/src/main/resources/stacks/HDP/2.2/services/OOZIE/configuration/oozie-env.xml
* ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.4.xml
* ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.3.xml
* ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/FixLzoCodecPathTest.java
* ambari-server/src/main/java/org/apache/ambari/server/checks/HardcodedStackVersionPropertiesCheck.java
* ambari-server/src/main/resources/common-services/OOZIE/4.2.0.2.3/configuration/oozie-env.xml
* ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
* ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
* ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.4.xml
* ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.4.xml
* ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml
* ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
* ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/OozieConfigCalculationTest.java
* ambari-server/src/test/java/org/apache/ambari/server/checks/HardcodedStackVersionPropertiesCheckTest.java
* ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.4.xml
* ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.4.xml
* ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
* ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml
* ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.2.xml
* ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-env.xml
* ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.4.xml
* ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FixLzoCodecPath.java
* ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/OozieConfigCalculation.java
* ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml


> MapReduce, Oozie service check failures after upgrade finished
> --------------------------------------------------------------
>
>                 Key: AMBARI-14764
>                 URL: https://issues.apache.org/jira/browse/AMBARI-14764
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.2.0
>            Reporter: Dmitry Lysnichenko
>            Assignee: Dmitry Lysnichenko
>             Fix For: 2.2.2
>
>         Attachments: AMBARI-14764.patch
>
>
> # MapReduce jobs were failing after upgrading and restarting MapReduce2 service (ClassNotFound
exception). Also Oozie job were failing because of the same reason.
> # This was caused because mapreduce.application.classpath property had the older version
of the LZO codec (/usr/hdp/current/hadoop/lib/hadoop-lzo-0.6.0.<OLD-STACK-VERSION>.jar).
> # During upgrade the path was valid because we had not switched /usr/hdp/current/hadoop
symlink to new version. After upgrade, once the symlink is updated to new stack version and
after restarting MapReduce2 this path becomes invalid.
> # To fix this updated the LZO codec path to /usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar
in mapreduce.application.classpath config property. Also in oozie-env property added export
HADOOP_OPTS="-Dhdp.version=$HDP_VERSION $HADOOP_OPTS”
> Fix:
> # During stack upgrade, update lzo codec path in mapreduce.application.classpath to
> {code}
> /usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar
> {code}
> Also, looks like we also need to update following property for TEZ.
> {code}
> <property>
> <name>tez.cluster.additional.classpath.prefix</name>
> <value>/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:/etc/hadoop/conf/secure</value>
> <description></description>
> </property>
> {code}
> # Update oozie-env and add
> {code}
> export HADOOP_OPTS="-Dhdp.version=$HDP_VERSION $HADOOP_OPTS"
> {code}
> # Add upgrade pre-checks to validate if any config properties have old-stack-version
hardcodings and report these config properties to be updated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message