Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============3017101279697044536=="
MIME-Version: 1.0
Subject: Re: Review Request 29614: Oozie service components should indicate
 security state
From: "Jaimin Jetly" <jaimin@hortonworks.com>
To: "Alejandro Fernandez" <afernandez@hortonworks.com>,
 "Andrew Onischuk" <aonishuk@hortonworks.com>,
 "Jaimin Jetly" <jaimin@hortonworks.com>,
 "Yusaku Sako" <yusaku@hortonworks.com>
Cc: "Robert Levas" <rlevas@hortonworks.com>, "Ambari" <dev@ambari.apache.org>
Date: Wed, 07 Jan 2015 04:03:41 -0000
Message-ID: <20150107040341.26560.47204@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Jaimin Jetly" <noreply@reviews.apache.org>
References: <20150107000355.26561.1417@reviews.apache.org>
In-Reply-To: <20150107000355.26561.1417@reviews.apache.org>
Reply-To: "Jaimin Jetly" <jaimin@hortonworks.com>

--===============3017101279697044536==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


> On Jan. 7, 2015, 12:03 a.m., Jaimin Jetly wrote:
> > ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py, line 90
> > <https://reviews.apache.org/r/29614/diff/1/?file=807394#file807394line90>
> >
> >     Looks like there is an existing bug in oozie stack scripts.
> >     Please either revert this change and add a TODO note and file a bug, or address it by removing all occurrences of "oozie_keytab" and "kinit_if_needed" (in oozie_service.py)
> 
> Robert Levas wrote:
>     `kinit_if_needed` is used to execute a command in `oozie_service.py`
>     
>     ```
>         cmd2 =  format("{kinit_if_needed} {put_shared_lib_to_hdfs_cmd} ; hadoop --config {hadoop_conf_dir} dfs -chmod -R 755 {oozie_hdfs_user_dir}/share")
>         ...
>         Execute( cmd2,
>           user = params.oozie_user,
>           not_if = format("{kinit_if_needed} hadoop --config {hadoop_conf_dir} dfs -ls /user/oozie/share | awk 'BEGIN {{count=0;}} /share/ {{count++}} END {{if (count > 0) {{exit 0}} else {{exit 1}}}}'"),
>           path = params.execute_path
>         )
>     ```
>     
>     Does this command go away too?
> 
> Jaimin Jetly wrote:
>     {kinit_if_needed} part in cmd2 commmand goes away. 
>     Ambari agent script should not kinit/authenticate with oozie service credentials on behalf of the oozie service. 
>     oozie service when it comes up is supposed to authenticate itself.
> 
> Robert Levas wrote:
>     Seems there is more to do than simply remove `kinit_if_needed`, any ideas?
>     
>     ```
>     2015-01-07 03:37:08,238 - Error while executing command 'start':
>     Traceback (most recent call last):
>       File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 149, in execute
>         method(env)
>       File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py", line 46, in start
>         oozie_service(action='start')
>       File "/var/lib/ambari-agent/cache/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py", line 61, in oozie_service
>         path = params.execute_path
>       File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
>         self.env.run()
>       File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 151, in run
>         self.run_action(resource, action)
>       File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 117, in run_action
>         provider_action()
>       File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 265, in action_run
>         raise ex
>     Fail: Execution of '/usr/hdp/current/oozie-client/bin/oozie-setup.sh sharelib create -fs hdfs://c6501.ambari.apache.org:8020 -locallib /usr/hdp/current/oozie-client/share ; hadoop --config /etc/hadoop/conf dfs -chmod -R 755 /user/oozie/share' returned 1.   setting OOZIE_CONFIG=${OOZIE_CONFIG:-/etc/oozie/conf}
>     ```

I will have to look into oozie logs for fiurther debugging.
If it is creating issues, I will suggest not to club this change with the patch.
We can file a bug specific to this issue and address it separately. 
Write a TODO note or file a bug.


- Jaimin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29614/#review66953
-----------------------------------------------------------


On Jan. 6, 2015, 2:31 a.m., Robert Levas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29614/
> -----------------------------------------------------------
> 
> (Updated Jan. 6, 2015, 2:31 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, Jaimin Jetly, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-8487
>     https://issues.apache.org/jira/browse/AMBARI-8487
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> The Oozie service components should indicate security state when queried by Ambari Agent via STATUS_COMMAND.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py f07e36d 
>   ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py c03f944 
>   ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params.py 3755666 
>   ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/status_params.py a665449 
>   ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 1fa2677 
> 
> Diff: https://reviews.apache.org/r/29614/diff/
> 
> 
> Testing
> -------
> 
> Manually tested in test cluster.
> 
> #Local test results: (Jenkins isn't working properly at this time):
> 
> Running tests for stack:2.0.6 service:OOZIE
> test_configure_default (test_oozie_client.TestOozieClient) ... ok
> test_configure_default_hdp22 (test_oozie_client.TestOozieClient) ... ok
> test_configure_secured (test_oozie_client.TestOozieClient) ... ok
> test_configure_default (test_oozie_server.TestOozieServer) ... ok
> test_configure_secured (test_oozie_server.TestOozieServer) ... ok
> **test_security_status (test_oozie_server.TestOozieServer) ... ok**
> test_start_default (test_oozie_server.TestOozieServer) ... ok
> test_start_secured (test_oozie_server.TestOozieServer) ... ok
> test_stop_default (test_oozie_server.TestOozieServer) ... ok
> test_stop_secured (test_oozie_server.TestOozieServer) ... ok
> test_service_check_default (test_service_check.TestServiceCheck) ... ok
> test_service_check_secured (test_service_check.TestServiceCheck) ... ok
> 
> ----------------------------------------------------------------------
> Ran 12 tests in 0.162s
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 28:50.695s
> [INFO] Finished at: Mon Jan 05 21:01:27 EST 2015
> [INFO] Final Memory: 41M/490M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Robert Levas
> 
>


--===============3017101279697044536==--