ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 41273: [DRAFT] AMBARI-14307. RU: Kafka broker restart failed on downgrade from HDP 2.3 to 2.2 due to circular symlink of /etc/hadoop/conf
Date Sun, 13 Dec 2015 23:02:41 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41273/#review110142
-----------------------------------------------------------



ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py (line
295)
<https://reviews.apache.org/r/41273/#comment169910>

    Please review this logic since it matters during RU downgrade and EU.



ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py (line
460)
<https://reviews.apache.org/r/41273/#comment169912>

    Before, on a host without hadoop components, it would fail on the "select" call and not
execute the rest of the code. Now, the /etc/hadoop/conf symlink is pointed to the /etc/hadoop/conf.backup
folder



ambari-common/src/main/python/resource_management/libraries/script/script.py (line 304)
<https://reviews.apache.org/r/41273/#comment169909>

    Called by conf_select.py to determine which hadoop conf dir to use.



ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py
(line 54)
<https://reviews.apache.org/r/41273/#comment169913>

    This was a bug in Kafka.


- Alejandro Fernandez


On Dec. 13, 2015, 10:58 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41273/
> -----------------------------------------------------------
> 
> (Updated Dec. 13, 2015, 10:58 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush
Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-14307
>     https://issues.apache.org/jira/browse/AMBARI-14307
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> * Ambari 2.2 at hash for AMBARI-14237 (e15d1e0fb3777f79cb473cf5de0f7a6bf30e63bd)
> * HDP 2.2.8.0 with HDFS, YARN/MR, ZK, Kafka (this host does not contain any hadoop components)
> * RU from  to 2.3.4.0 and reach the Finalize screen.
> * Then downgrade,
> 
> Kafka broker restart failed on downgrade
> ```
> Traceback (most recent call last):
>   File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\",
line 39, in <module>
>     BeforeStartHook().execute()
>   File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
line 217, in execute
>     method(env)
>   File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py\",
line 36, in hook
>     create_topology_script_and_mapping()
>   File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\",
line 69, in create_topology_script_and_mapping
>     create_topology_mapping()
>   File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py\",
line 36, in create_topology_mapping
>     group=params.user_group) 
>   File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\", line 154,
in __init__
>     self.env.run()
>   File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 158, in run
>     self.run_action(resource, action)
>   File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 121, in run_action
>     provider_action()
>   File \"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py\",
line 144, in action_create
>     raise Fail(\"Applying %s failed, looped symbolic links found while resolving %s\"
% (self.resource, path))
> resource_management.core.exceptions.Fail: Applying Directory['/etc/hadoop/conf'] failed,
looped symbolic links found while resolving /etc/hadoop/conf
> Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py',
'START', '/var/lib/ambari-agent/data/command-2873.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-START',
'/var/lib/ambari-agent/data/structured-out-2873.json', 'INFO', '/var/lib/ambari-agent/tmp']
> ```
> 
> 
> This failed because /etc/hadoop/conf is now a circular symlink
> 
> 
> Diffs
> -----
> 
>   ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
7fd64b5 
>   ambari-common/src/main/python/resource_management/libraries/script/script.py 398cee8

>   ambari-server/src/main/resources/common-services/KAFKA/0.8.1.2.2/package/scripts/kafka_broker.py
6fcf08a 
>   ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 85d0d25

>   ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py f8b38db 
> 
> Diff: https://reviews.apache.org/r/41273/diff/
> 
> 
> Testing
> -------
> 
> Tested on the following code paths,
> 
> 1. Fresh install of HDP 2.1
> 2. Installed bits for HDP 2.3 and verified the conf folder structure
> 3. EU to HDP 2.3
> 4. Installed HDFS client on the host that didn't have /etc/hadoop/conf and then it symlinked
it to /usr/hdp/current/hadoop-client/conf
> 
> 
> 1. Fresh install of HDP 2.2
> 2. Installed bits for HDP 2.3 and verfied the conf folder structure
> 3. RU to HDP 2.3 and downgrade
> 
> Still need to fix unit tests.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message