ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 40688: AMBARI-14056. Stack Upgrade from HDP 2.1 or 2.2 to 2.3 to preserve configs for AppTimelineServer ATS version 1.0
Date Wed, 25 Nov 2015 19:39:23 GMT


> On Nov. 25, 2015, 3:56 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml,
lines 243-244
> > <https://reviews.apache.org/r/40688/diff/1/?file=1139543#file1139543line243>
> >
> >     execute-stage implementation (ExecuteStage) has only one Task defined, not a
list, so is the first one getting ignored now?

You're right, only one task per ExecuteStage


> On Nov. 25, 2015, 3:56 p.m., Nate Cole wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml, lines
251-252
> > <https://reviews.apache.org/r/40688/diff/1/?file=1139546#file1139546line251>
> >
> >     This is incorrect for HDP-2.3.4 based on how that property is defined on the
stack.

Whenever upgrading from 2.2 -> 2.3.* (in either 2.3.0.0 or 2.3.4.0), we shouldn't change
the configs for ATS.
If they upgrade to HDP 2.3.0.0, then those bits still have ATS 1.0, whose configs are the
same as HDP 2.2 stack!
If they upgrade to HDP 2.3.4.0, then those bits support both ATS 1.0 and 1.5, but the existing
ATS must continue to work the current configs (from HDP 2.2 stack).

If a customer uses Ambari 2.1.3 to deploy HDP 2.3.4.0, only then will they have ATS 1.5 with
the 2.3 configs.


- Alejandro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40688/#review108013
-----------------------------------------------------------


On Nov. 25, 2015, 7:36 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40688/
> -----------------------------------------------------------
> 
> (Updated Nov. 25, 2015, 7:36 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Swapan Shridhar,
and Vinod Kumar Vavilapalli.
> 
> 
> Bugs: AMBARI-14056
>     https://issues.apache.org/jira/browse/AMBARI-14056
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> HDP 2.3.4.0 has support for both AppTimelineServer v1.0 and v1.5
> The HDP 2.3 stack definition will make ATS v1.5 the default for configs.
> This means that any stack upgrade that modifies the configs and upgrades to version x
where 2.3.0.0 <= x < 2.3.4.0 will fail since those binaries only support ATS v1.0 but
the configs are for ATS 1.5.
> 
> The fix is to modify the following upgrade packs
> HDP 2.1 -> 2.3 in nonrolling
> HDP 2.2 -> 2.3 in both rolling and nonrolling
> 
> So that they preserve the ATS v1.0 configs.
> For tez-site, keep
> ```
> tez.history.logging.service.class = org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService
> ```
> 
> For yarn-site, keep 
> ```
> yarn.timeline-service.version = 1.0
> yarn.timeline-service.store-class = org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore
> ```
> and it's ok to add the HDP 2.3 configs that are not being used, e.g.,
> ```
> yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes = org.apache.tez.dag.history.logging.ats.TimelineCachePluginImpl
> yarn.timeline-service.entity-group-fs-store.summary-store = org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore
> yarn.timeline-service.entity-group-fs-store.scan-interval-seconds = 60
> yarn.timeline-service.entity-group-fs-store.cleaner-interval-seconds = 3600
> yarn.timeline-service.entity-group-fs-store.retain-seconds = 604800
> yarn.timeline-service.entity-group-fs-store.active-dir = /ats/active/
> yarn.timeline-service.entity-group-fs-store.done-dir = /ats/done/
> ```
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml
91c46961 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml
beb316b 
>   ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 6a85479 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml ecfa031

> 
> Diff: https://reviews.apache.org/r/40688/diff/
> 
> 
> Testing
> -------
> 
> Tested the following combinations on a cluster with HDFS, YARN, MR, ZK, and NameNode
HA.
> 
> HDP 2.2.4.2 -> 2.3.2.0
>   Express Upgrade without Tez
>   Rolling Upgrade with Tez
>   
> HDP 2.2.4.2 -> 2.3.4.0
>   Rolling Upgrade withtout Tez
>   Express Upgrade with Tez
>   
> Python unit tests passed,
> ----------------------------------------------------------------------
> Total run:836
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message