ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk" <aonis...@hortonworks.com>
Subject Re: Review Request 33864: CoordinatorStoreService Removed From Oozie
Date Wed, 06 May 2015 15:15:30 GMT


> On May 6, 2015, 2:22 p.m., Andrew Onischuk wrote:
> > Looks good. 
> > 
> > For other services we put new configurations into HDP newer stacks. 
> > I think we should do in common services in the same way as here, that way new stacks
will also be compitable with other vendors.
> 
> Robert Levas wrote:
>     I agree. I think I am on the hook to create a Wiki page on how to properly utilize
common services like this.

Please ping me once you have that, it's interesting to see. Also I've got some tasks on optimizing
the inheritance. If Jayush agrees on this, I think I can move the HDP things to common services,
in scope of that. 

Btw, Also I'm looking forward to remove the checks for stack_version >= "2.3" or like this.
By replacing that with inherited configurations from -env's. Which will make the scripts 100%
compitable with other vendors.


- Andrew


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33864/#review82675
-----------------------------------------------------------


On May 6, 2015, 2:04 p.m., Robert Levas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33864/
> -----------------------------------------------------------
> 
> (Updated May 6, 2015, 2:04 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jonathan Hurley, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-10919
>     https://issues.apache.org/jira/browse/AMBARI-10919
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> After installing Oozie, the Oozie service check fails with the below error. 
> 
> ```
> 2015-05-04 10:19:15,281 - Error while executing command 'service_check':
> Traceback (most recent call last):
>   File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 214, in execute
>     method(env)
>   File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py",
line 44, in service_check
>     OozieServiceCheckDefault.oozie_smoke_shell_file(smoke_test_file_name)
>   File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/service_check.py",
line 69, in oozie_smoke_shell_file
>     logoutput=True
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148,
in __init__
>     self.env.run()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line
152, in run
>     self.run_action(resource, action)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line
118, in run_action
>     provider_action()
>   File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
line 269, in action_run
>     raise ex
> Fail: Execution of '/var/lib/ambari-agent/data/tmp/oozieSmoke2.sh redhat /usr/hdp/current/oozie-client
/etc/oozie/conf /usr/hdp/current/oozie-client/bin /etc/hadoop/conf /usr/hdp/current/hadoop-client/bin
ambari-qa True /etc/security/keytabs/smokeuser.headless.keytab /usr/bin/kinit ambari-qa@EXAMPLE.COM'
returned 1. Located Oozie examples JAR at /usr/hdp/current/oozie-client/doc
> 15/05/04 10:18:20 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion
interval = 360 minutes, Emptier interval = 0 minutes.
> Moved: 'hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/examples'
to trash at: hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/.Trash/Current
> 15/05/04 10:18:24 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion
interval = 360 minutes, Emptier interval = 0 minutes.
> Moved: 'hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/input-data'
to trash at: hdfs://os-hdp-2-3-amb-r6-1430724014-sec-knox-4.novalocal:8020/user/ambari-qa/.Trash/Current
> /usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa@EXAMPLE.COM;
source /etc/oozie/conf/oozie-env.sh ; /usr/hdp/current/oozie-client/bin/oozie -Doozie.auth.token.cache=false
job -oozie http://os-hdp-2-3-amb-r6-1430724014-sec-knox-3.novalocal:11000/oozie -config /usr/hdp/current/oozie-client/doc/examples/apps/map-reduce/job.properties
-run
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 1 sec. Retry count = 1
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 2 sec. Retry count = 2
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 4 sec. Retry count = 3
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 8 sec. Retry count = 4
> Connection exception has occurred [ java.net.ConnectException Connection refused ]. Trying
after 16 sec. Retry count = 5
> Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server. No of retries
= 5. Exception = Connection refused
> Invalid sub-command: Missing argument for option: info
> 
> use 'help [sub-command]' for help details
> Invalid sub-command: Missing argument for option: info
> 
> use 'help [sub-command]' for help details
> 
> workflow_status=
> ```
> This is due to the following ClassNotFoundException when starting up Oozie server:
> 
> ```
> org.apache.oozie.service.ServiceException: E0103: Could not load service classes, java.lang.ClassNotFoundException:
Class org.apache.oozie.service.CoordinatorStoreService not found
>         at org.apache.oozie.service.Services.loadServices(Services.java:309)
>         at org.apache.oozie.service.Services.init(Services.java:213)
>         at org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.java:46)
>         at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4210)
>         at org.apache.catalina.core.StandardContext.start(StandardContext.java:4709)
>         at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
>         at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
>         at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
>         at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:675)
>         at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:601)
>         at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
>         at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
>         at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
>         at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
>         at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
>         at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
>         at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
>         at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
>         at org.apache.catalina.core.StandardService.start(StandardService.java:525)
>         at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
>         at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
>         at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.oozie.service.CoordinatorStoreService
not found
>         at org.apache.hadoop.conf.Configuration.getClasses(Configuration.java:2174)
>         at org.apache.oozie.service.ConfigurationService$LogChangesConfiguration.getClasses(ConfigurationService.java:376)
>         at org.apache.oozie.service.ConfigurationService.getClasses(ConfigurationService.java:533)
>         at org.apache.oozie.service.Services.loadServices(Services.java:288)
>         ... 26 more
> Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.service.CoordinatorStoreService
not found
>         at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
>         at org.apache.oozie.util.XConfiguration.getClassByName(XConfiguration.java:194)
>         at org.apache.hadoop.conf.Configuration.getClasses(Configuration.java:2170)
>         ... 29 more
> ```
> 
> Which is caused by an outdated Oozie service being declared in oozie-site.xml - https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/configuration/oozie-site.xml#L262
> 
> **NOTE:** A new common services version for OOZIE was created - 5.0.0.2.3 (though it
seems like it should have been labeled 4.1.0.2.3, looking at the log file).  This was copied
from the 4.0.0.2.0 version and the oozie-site.xml and alerts.json files were adjusted as needed.
 The referece to OOZIE from HDP 2.3 was altered to point to common_services/OOZIE/5.0.0.2.3.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/alerts.json PRE-CREATION

>   ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/configuration/oozie-site.xml
PRE-CREATION 
>   ambari-server/src/main/resources/common-services/OOZIE/5.0.0.2.3/metainfo.xml PRE-CREATION

>   ambari-server/src/main/resources/stacks/HDP/2.3.GlusterFS/services/OOZIE/metainfo.xml
3df7d58 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/OOZIE/metainfo.xml 9dcff94

>   ambari-server/src/test/java/org/apache/ambari/server/stack/KerberosDescriptorTest.java
f60668f 
> 
> Diff: https://reviews.apache.org/r/33864/diff/
> 
> 
> Testing
> -------
> 
> Manually tested in clusters using HDP 2.0 through HDP 2.3.
> 
> Added new unit tests for common_services/OOZIE/5.0.0.2.3
> 
> **Jenkins test results: PENDING**
> 
> 
> Thanks,
> 
> Robert Levas
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message