ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hurley" <jhur...@hortonworks.com>
Subject Re: Review Request 42337: AMBARI-14678. RU - Oozie HA, Service Check fails since the pre-upgrade directive has multiple tasks that should run only on the specified hosts
Date Fri, 15 Jan 2016 18:22:06 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42337/#review114740
-----------------------------------------------------------


Upon thinking about this more with Alejandro, it seems like we don't really want to wrap all
kinds of tasks inside of the <pre-upgrade> in their own stage. Consider the cases where
you have 10 `configure` type tasks. Those already run sequentially b/c they are server-side
tasks and can be grouped into a single stage. The main problem is that we have mixtures inside
of `<pre-upgrade>` of `configure` and `execute` types.

I think the cleanest way is to annotate tasks which _must_ be sequential like so:

```
        <pre-upgrade>
          <task xsi:type="configure" id="hdp_2_3_0_0_oozie_remove_redundant_configurations"/>
          <task xsi:type="configure" id="hdp_2_3_0_0_oozie_remove_redundant_configurations"/>
          <task xsi:type="configure" id="hdp_2_3_0_0_oozie_remove_redundant_configurations"/>
          
          <task xsi:type="execute" hosts="all" synchronized="true" summary="Shut down all
Oozie servers">
            <script>scripts/oozie_server.py</script>
            <function>stop</function>
          </task>

          <task xsi:type="execute" hosts="any" synchronized="true" summary="Upgrading the
database and creating a new sharelib">
            <script>scripts/oozie_server_upgrade.py</script>
            <function>upgrade_oozie_database_and_sharelib</function>
          </task>
        </pre-upgrade>
```

We can group all of the tasks which are not `synchronized=true` in a single stage, while creating
new stages for all tasks which are synchronized.

- Jonathan Hurley


On Jan. 14, 2016, 9:45 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42337/
> -----------------------------------------------------------
> 
> (Updated Jan. 14, 2016, 9:45 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush
Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-14678
>     https://issues.apache.org/jira/browse/AMBARI-14678
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> STR:
> HDP 2.2
> Multiple Oozie Servers
> RU to HDP 2.3
> 
> The pre-upgrade directive for Oozie has 
> ```
> <pre-upgrade>
>   <task xsi:type="execute" hosts="all" summary="Shut down all Oozie servers">
>     <script>scripts/oozie_server.py</script>
>     <function>stop</function>
>   </task>
> 
>   <task xsi:type="execute" hosts="any" summary="Upgrading the database and creating
a new sharelib">
>     <script>scripts/oozie_server_upgrade.py</script>
>     <function>upgrade_oozie_database_and_sharelib</function>
>   </task>
> </pre-upgrade>
> ```
> 
> Notice that the first task runs on all hosts, and the second task runs on exactly one
random host. The problem is that the scheduler runs both tasks in one stage on the union of
these hosts. Therefore, the 2nd task runs on the wrong set of hosts.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
aa852bd 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java
b0df83f 
>   ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 7e61942

>   ambari-server/src/test/resources/stacks/HDP/2.0.7/services/OOZIE/metainfo.xml PRE-CREATION

>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test.xml 4f71b4a

> 
> Diff: https://reviews.apache.org/r/42337/diff/
> 
> 
> Testing
> -------
> 
> Verified during RU from HDP 2.2 to 2.3 with multiple Oozie Servers.
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message