ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Levas" <rle...@hortonworks.com>
Subject Review Request 33864: CoordinatorStoreService Removed From Oozie
Date Tue, 05 May 2015 21:13:05 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33864/
-----------------------------------------------------------

Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Sumit Mohanty.


Bugs: AMBARI-10919
    https://issues.apache.org/jira/browse/AMBARI-10919


Repository: ambari


Description
-------

After installing Oozie, the Oozie service check fails with the below error. 

```
2015-05-04 10:19:15,281 - Error while executing command 'service_check':
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 214, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py",
line 44, in service_check
    OozieServiceCheckDefault.oozie_smoke_shell_file(smoke_test_file_name)
  File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py",
line 69, in oozie_smoke_shell_file
    logoutput=True
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152,
in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118,
in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line
269, in action_run
    raise ex
Fail: Execution of '/var/lib/ambari-agent/data/tmp/oozieSmoke2.sh redhat /usr/hdp/current/oozie-client
/etc/oozie/conf /usr/hdp/current/oozie-client/bin /etc/hadoop/conf /usr/hdp/current/hadoop-client/bin
ambari-qa True /etc/security/keytabs/smokeuser.headless.keytab /usr/bin/kinit ambari-qa@EXAMPLE.COM'
returned 1. Located Oozie examples JAR at /usr/hdp/current/oozie-client/doc
15/05/04 10:18:20 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval
= 360 minutes, Emptier interval = 0 minutes.
Moved: 'hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/examples'
to trash at: hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/.Trash/Current
15/05/04 10:18:24 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval
= 360 minutes, Emptier interval = 0 minutes.
Moved: 'hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/input-data'
to trash at: hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/.Trash/Current
/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM;
source /etc/oozie/conf/oozie-env.sh ; /usr/hdp/current/oozie-client/bin/oozie -Doozie.auth.token.cache=false
job -oozie http://os-hdp-2-3-amb-r6-1430724014-sec-knox-3.novalocal:11000/oozie -config /usr/hdp/current/oozie-client/doc/examples/apps/map-reduce/job.properties
-run
Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 1 sec. Retry count = 1
Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 2 sec. Retry count = 2
Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 4 sec. Retry count = 3
Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 8 sec. Retry count = 4
Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 16 sec. Retry count = 5
Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries
= 5. Exception = Connection refused
Invalid sub-command: Missing argument for option: info

use 'help [sub-command]' for help details
Invalid sub-command: Missing argument for option: info

use 'help [sub-command]' for help details

workflow_status=
```
This is due to the following ClassNotFoundException when starting up Oozie server:

```
org.apache.oozie.service.ServiceException: E0103: Could not load service classes, java.lang.ClassNotFoundException:
Class org.apache.oozie.service.CoordinatorStoreService not found
        at org.apache.oozie.service.Services.loadServices(Services.java:309)
        at org.apache.oozie.service.Services.init(Services.java:213)
        at org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
        at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
        at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:675)
        at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:601)
        at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
        at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
        at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
        at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
        at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
        at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
        at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
        at org.apache.catalina.core.StandardService.start(StandardService.java:525)
        at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
        at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
        at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.oozie.service.CoordinatorStoreService
not found
        at org.apache.hadoop.conf.Configuration.getClasses(Configuration.java:2174)
        at org.apache.oozie.service.ConfigurationService$LogChangesConfiguration.getClasses(ConfigurationService.java:376)
        at org.apache.oozie.service.ConfigurationService.getClasses(ConfigurationService.java:533)
        at org.apache.oozie.service.Services.loadServices(Services.java:288)
        ... 26 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.service.CoordinatorStoreService
not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
        at org.apache.oozie.util.XConfiguration.getClassByName(XConfiguration.java:194)
        at org.apache.hadoop.conf.Configuration.getClasses(Configuration.java:2170)
        ... 29 more
```

Which is caused by an outdated Oozie service being declared in oozie-site.xml - https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-site.xml#L262

**NOTE:** A new common services version for OOZIE was created - 5.0.0.2.3 (though it seems
like it should have been labeled 4.1.0.2.3, looking at the log file).  This was copied from
the 4.0.0.2.0 version and the oozie-site.xml and alerts.json files were adjusted as needed.
 The referece to OOZIE from HDP 2.3 was altered to point to common_services/OOZIE/5.0.0.2.3.


Diffs
-----

  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/alerts.json PRE-CREATION

  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/configuration/oozie-env.xml
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/configuration/oozie-log4j.xml
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/configuration/oozie-site.xml
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/kerberos.json PRE-CREATION

  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/metainfo.xml PRE-CREATION

  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/alerts/alert_check_oozie_server.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/files/oozieSmoke2.sh
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/files/wrap_ooziedb.sh
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/check_oozie_server_status.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/oozie.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/oozie_client.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/oozie_server.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/oozie_server_upgrade.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/oozie_service.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/params.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/params_linux.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/params_windows.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/service_check.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/scripts/status_params.py
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/templates/adminusers.txt.j2
PRE-CREATION 
  ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/package/templates/oozie-log4j.properties.j2
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/metainfo.xml 9dcff94 
  ambari-server/src/test/java/org/apache/ambari/server/stack/KerberosDescriptorTest.java f60668f

  ambari-server/src/test/python/stacks/2.3/OOZIE/test_oozie_client.py PRE-CREATION 
  ambari-server/src/test/python/stacks/2.3/OOZIE/test_oozie_server.py PRE-CREATION 
  ambari-server/src/test/python/stacks/2.3/OOZIE/test_service_check.py PRE-CREATION 

Diff: https://reviews.apache.org/r/33864/diff/


Testing
-------

Manually tested in clusters using HDP 2.0 through HDP 2.3.

Added new unit tests for common_services/OOZIE/5.0.0.2.3

**Jenkins test results: PENDING**


Thanks,

Robert Levas


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message