ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nate Cole" <nc...@hortonworks.com>
Subject Re: Review Request 42038: AMBARI-14581. EU - orchestration of Hive Server and MetaStore needs to be sequential
Date Mon, 11 Jan 2016 13:29:23 GMT


> On Jan. 8, 2016, 8:49 a.m., Jonathan Hurley wrote:
> > Why go through all of these code changes to accomplish this? It seems like you could
just have removed the `parallel-scheduler` element from the upgrade XML:
> > 
> > ```
> >     <group xsi:type="restart" name="HIVE" title="Hive">
> >       <service-check>false</service-check>
> >       <skippable>true</skippable>
> >       <supports-auto-skip-failure>false</supports-auto-skip-failure>
> >       <parallel-scheduler/>
> >       <service name="HIVE">
> >         <component>HIVE_METASTORE</component>
> >         <component>HIVE_SERVER</component>
> >         <component>WEBHCAT_SERVER</component>
> >         <component>HIVE_CLIENT</component>
> >         <component>HCAT</component>
> >       </service>
> >     </group>
> > ```
> 
> Nate Cole wrote:
>     +1 for this
> 
> Alejandro Fernandez wrote:
>     I want the group to still be in parallel, but only Hive Server and WebHCat server
to be serial. Breaking it up into 2 separate groups seemed too clunky. If the default behavior
of parallel-scheduler changes, then I still ensure that those 2 components run sequentially.
> 
> Jonathan Hurley wrote:
>     That's what the groupings are for though. I'd expect this kind of change to be an
XML-only change since it's so simple. Seems like we're introducing a lot of new code which
overcomplicates the solution.
> 
> Alejandro Fernandez wrote:
>     I thought about that, but didn't want the EU/RU wizard to then show multiple groups
for Hive.
> 
> Jonathan Hurley wrote:
>     I think this is a matter of trading off between a cleaner XML with a single group
vs having additional complex code for calculating the maximum degree of parallelism. My feeling
is that even though it's a neat feature, it's not something required to fix this problem.
Fixing it in the XML seems to be more intuitive since it will be represented to the user as
2 different groups. 
>     
>     If others feel strongly that we should keep Hive as a single group, then I'm OK with
it. I just didn't think this much code was necessary to fix the problem.

I would prefer keeping the code simpler.  The whole point of the upgrade pack "verbosity"
is to make it clear to someone viewing it what is going to happen.  That there's two hive
groups being shown isn't a big deal.  "Hive" and "Hive Clients."


- Nate


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42038/#review113472
-----------------------------------------------------------


On Jan. 7, 2016, 5:18 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42038/
> -----------------------------------------------------------
> 
> (Updated Jan. 7, 2016, 5:18 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan Hurley, Jayush
Luniya, and Nate Cole.
> 
> 
> Bugs: AMBARI-14581
>     https://issues.apache.org/jira/browse/AMBARI-14581
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> During an Express Upgrade, all of the tasks for Hive Server and Hive MetaStore run in
parallel, so if these components have multiple hosts, then running in parallel can cause errors
during the upgrade.
> E.g., Hive Server copies tarballs, which can only run one at a time. Hive MetaStore updates
the schema, and only one can run at a time.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java
aa852bd 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java
b0df83f 
>   ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Task.java
60091a6 
>   ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml
a46a5cd 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml
e57b13f 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
b1c3739 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml
ba2578b 
>   ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java 7e61942

>   ambari-server/src/test/java/org/apache/ambari/server/state/stack/UpgradePackTest.java
15be8b4 
>   ambari-server/src/test/resources/stacks/HDP/2.0.7/services/HIVE/metainfo.xml 35a4d05

>   ambari-server/src/test/resources/stacks/HDP/2.0.7/services/YARN/metainfo.xml beee6a0

>   ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test_nonrolling.xml
9f57ca8 
> 
> Diff: https://reviews.apache.org/r/42038/diff/
> 
> 
> Testing
> -------
> 
> Ran unit tests in UpgradeHelperTest.java and UpgradePackTest.java
> Verified with EU from HDP 2.2->2.3 with multiple Hive MetaStores.
> 
> Waiting for unit test results.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message