ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 40688: AMBARI-14056. Stack Upgrade from HDP 2.1 or 2.2 to 2.3 to preserve configs for AppTimelineServer ATS version 1.0
Date Wed, 25 Nov 2015 19:36:13 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40688/
-----------------------------------------------------------

(Updated Nov. 25, 2015, 7:36 p.m.)


Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Swapan Shridhar, and
Vinod Kumar Vavilapalli.


Changes
-------

Addressed issues


Bugs: AMBARI-14056
    https://issues.apache.org/jira/browse/AMBARI-14056


Repository: ambari


Description
-------

HDP 2.3.4.0 has support for both AppTimelineServer v1.0 and v1.5
The HDP 2.3 stack definition will make ATS v1.5 the default for configs.
This means that any stack upgrade that modifies the configs and upgrades to version x where
2.3.0.0 <= x < 2.3.4.0 will fail since those binaries only support ATS v1.0 but the
configs are for ATS 1.5.

The fix is to modify the following upgrade packs
HDP 2.1 -> 2.3 in nonrolling
HDP 2.2 -> 2.3 in both rolling and nonrolling

So that they preserve the ATS v1.0 configs.
For tez-site, keep
```
tez.history.logging.service.class = org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService
```

For yarn-site, keep 
```
yarn.timeline-service.version = 1.0
yarn.timeline-service.store-class = org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore
```
and it's ok to add the HDP 2.3 configs that are not being used, e.g.,
```
yarn.timeline-service.entity-group-fs-store.group-id-plugin-classes = org.apache.tez.dag.history.logging.ats.TimelineCachePluginImpl
yarn.timeline-service.entity-group-fs-store.summary-store = org.apache.hadoop.yarn.server.timeline.RollingLevelDBTimelineStore
yarn.timeline-service.entity-group-fs-store.scan-interval-seconds = 60
yarn.timeline-service.entity-group-fs-store.cleaner-interval-seconds = 3600
yarn.timeline-service.entity-group-fs-store.retain-seconds = 604800
yarn.timeline-service.entity-group-fs-store.active-dir = /ats/active/
yarn.timeline-service.entity-group-fs-store.done-dir = /ats/done/
```


Diffs (updated)
-----

  ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml 91c46961

  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml beb316b

  ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/upgrade-2.3.xml 6a85479 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml ecfa031 

Diff: https://reviews.apache.org/r/40688/diff/


Testing
-------

Tested the following combinations on a cluster with HDFS, YARN, MR, ZK, and NameNode HA.

HDP 2.2.4.2 -> 2.3.2.0
  Express Upgrade without Tez
  Rolling Upgrade with Tez
  
HDP 2.2.4.2 -> 2.3.4.0
  Rolling Upgrade withtout Tez
  Express Upgrade with Tez
  
Python unit tests passed,
----------------------------------------------------------------------
Total run:836
Total errors:0
Total failures:0
OK


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message