ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Levas" <rle...@hortonworks.com>
Subject Re: Review Request 33864: CoordinatorStoreService Removed From Oozie
Date Wed, 06 May 2015 14:57:43 GMT


> On May 6, 2015, 10:22 a.m., Andrew Onischuk wrote:
> > Looks good. 
> > 
> > For other services we put new configurations into HDP newer stacks. 
> > I think we should do in common services in the same way as here, that way new stacks
will also be compitable with other vendors.

I agree. I think I am on the hook to create a Wiki page on how to properly utilize common
services like this.


- Robert


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33864/#review82675
-----------------------------------------------------------


On May 6, 2015, 10:04 a.m., Robert Levas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33864/
> -----------------------------------------------------------
> 
> (Updated May 6, 2015, 10:04 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-10919
>     https://issues.apache.org/jira/browse/AMBARI-10919
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> After installing Oozie, the Oozie service check fails with the below error. 
> 
> ```
> 2015-05-04 10:19:15,281 - Error while executing command 'service_check':
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 214, in execute
>     method(env)
>   File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py",
line 44, in service_check
>     OozieServiceCheckDefault.oozie_smoke_shell_file(smoke_test_file_name)
>   File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py",
line 69, in oozie_smoke_shell_file
>     logoutput=True
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148,
in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line
152, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line
118, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
line 269, in action_run
>     raise ex
> Fail: Execution of '/var/lib/ambari-agent/data/tmp/oozieSmoke2.sh redhat /usr/hdp/current/oozie-client
/etc/oozie/conf /usr/hdp/current/oozie-client/bin /etc/hadoop/conf /usr/hdp/current/hadoop-client/bin
ambari-qa True /etc/security/keytabs/smokeuser.headless.keytab /usr/bin/kinit ambari-qa@EXAMPLE.COM'
returned 1. Located Oozie examples JAR at /usr/hdp/current/oozie-client/doc
> 15/05/04 10:18:20 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion
interval = 360 minutes, Emptier interval = 0 minutes.
> Moved: 'hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/examples'
to trash at: hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/.Trash/Current
> 15/05/04 10:18:24 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion
interval = 360 minutes, Emptier interval = 0 minutes.
> Moved: 'hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/input-data'
to trash at: hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/.Trash/Current
> /usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM;
source /etc/oozie/conf/oozie-env.sh ; /usr/hdp/current/oozie-client/bin/oozie -Doozie.auth.token.cache=false
job -oozie http://os-hdp-2-3-amb-r6-1430724014-sec-knox-3.novalocal:11000/oozie -config /usr/hdp/current/oozie-client/doc/examples/apps/map-reduce/job.properties
-run
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 1 sec. Retry count = 1
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 2 sec. Retry count = 2
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 4 sec. Retry count = 3
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 8 sec. Retry count = 4
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 16 sec. Retry count = 5
> Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries
= 5. Exception = Connection refused
> Invalid sub-command: Missing argument for option: info
> 
> use 'help [sub-command]' for help details
> Invalid sub-command: Missing argument for option: info
> 
> use 'help [sub-command]' for help details
> 
> workflow_status=
> ```
> This is due to the following ClassNotFoundException when starting up Oozie server:
> 
> ```
> org.apache.oozie.service.ServiceException: E0103: Could not load service classes, java.lang.ClassNotFoundException:
Class org.apache.oozie.service.CoordinatorStoreService not found
>         at org.apache.oozie.service.Services.loadServices(Services.java:309)
>         at org.apache.oozie.service.Services.init(Services.java:213)
>         at org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
>         at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>         at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>         at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
>         at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
>         at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
>         at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:675)
>         at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:601)
>         at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>         at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
>         at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
>         at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>         at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
>         at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
>         at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
>         at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
>         at org.apache.catalina.core.StandardService.start(StandardService.java:525)
>         at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
>         at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>         at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.oozie.service.CoordinatorStoreService
not found
>         at org.apache.hadoop.conf.Configuration.getClasses(Configuration.java:2174)
>         at org.apache.oozie.service.ConfigurationService$LogChangesConfiguration.getClasses(ConfigurationService.java:376)
>         at org.apache.oozie.service.ConfigurationService.getClasses(ConfigurationService.java:533)
>         at org.apache.oozie.service.Services.loadServices(Services.java:288)
>         ... 26 more
> Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.service.CoordinatorStoreService
not found
>         at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
>         at org.apache.oozie.util.XConfiguration.getClassByName(XConfiguration.java:194)
>         at org.apache.hadoop.conf.Configuration.getClasses(Configuration.java:2170)
>         ... 29 more
> ```
> 
> Which is caused by an outdated Oozie service being declared in oozie-site.xml - https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-site.xml#L262
> 
> **NOTE:** A new common services version for OOZIE was created - 5.0.0.2.3 (though it
seems like it should have been labeled 4.1.0.2.3, looking at the log file).  This was copied
from the 4.0.0.2.0 version and the oozie-site.xml and alerts.json files were adjusted as needed.
 The referece to OOZIE from HDP 2.3 was altered to point to common_services/OOZIE/5.0.0.2.3.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/alerts.json PRE-CREATION

>   ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/configuration/oozie-site.xml
PRE-CREATION 
>   ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/metainfo.xml PRE-CREATION

>   ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/OOZIE/metainfo.xml
3df7d58 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/metainfo.xml 9dcff94

>   ambari-server/src/test/java/org/apache/ambari/server/stack/KerberosDescriptorTest.java
f60668f 
> 
> Diff: https://reviews.apache.org/r/33864/diff/
> 
> 
> Testing
> -------
> 
> Manually tested in clusters using HDP 2.0 through HDP 2.3.
> 
> Added new unit tests for common_services/OOZIE/5.0.0.2.3
> 
> **Jenkins test results: PENDING**
> 
> 
> Thanks,
> 
> Robert Levas
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message