ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sumit Mohanty" <smoha...@hortonworks.com>
Subject Re: Review Request 34735: Implement support in Yarn stack scripts to create directory for "yarn.timeline-service.leveldb-state-store.path".
Date Thu, 28 May 2015 01:05:26 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34735/#review85490
-----------------------------------------------------------

Ship it!


Ship It!

- Sumit Mohanty


On May 27, 2015, 9:33 p.m., Robert Nettleton wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34735/
> -----------------------------------------------------------
> 
> (Updated May 27, 2015, 9:33 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, John Speidel, Mahadev Konar, Robert Levas,
and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-11442
>     https://issues.apache.org/jira/browse/AMBARI-11442
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> This patch addresses AMBARI-11442. 
> 
> The Ambari stack scripts for Yarn currently have a problem regarding the startup of the
Yarn Application Timeline Service.
> 
> The timeline service has two properties for DB storage:
> 
> yarn.timeline-service.leveldb-timeline-store.path
> yarn.timeline-service.leveldb-state-store.path
> 
> The current Yarn stack scripts only create a directory for "yarn.timeline-service.leveldb-timeline-store.path"
during a cluster startup.
> 
> If the default settings are used for these properties, then things will work fine, since
both are mapped to the same root directory.
> 
> If the two properties are set to different values, then the App Timeline Service will
fail to startup, since the directory associated with "yarn.timeline-service.leveldb-state-store.path"
will not have been created.
> 
> The "yarn.timeline-service.leveldb-state-store.path" property was added in Ambari 2.1
in HDP 2.2. The rest of the Yarn stack code should be updated to use this variable to create
a separate directory, if needed.
> 
> This patch addresses the problem by implementing the following:
> 
> 1. Adds a new parameter to read in "yarn.timeline-service.leveldb-state-store.path" whenever
HDP 2.2 or greater is used (this new property was introduced in the HDP 2.2 stacks). 
> 2. Adds some code to yarn.py to create this new directory, whenever HDP 2.2 or greater
is selected as the stack.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
67b9565 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
d77654a 
> 
> Diff: https://reviews.apache.org/r/34735/diff/
> 
> 
> Testing
> -------
> 
> 1. Ran the Python unit tests (all passing).
> 2. Deployed a single-node Yarn cluster (HDP 2.3) with the Ambari UI, configuring these
two properties to point towards different directories.  Verified that the App Timeline Service
starts up properly (as well as the rest of Yarn), and that Ambari creates the two separate
directories.
> 3. Deployed a single-node Yarn cluster (HDP 2.3) with the Ambari UI, using all the defaults
provided by Ambari UI for Yarn config, and verified that the App Timeline Service starts up
properly (as well as the rest of Yarn).  
> 4. Deployed a single-node Yarn cluster with HDP 2.1 with the UI defaults.  This Yarn
deployment failed, but upon further investigation I found that this stack deployment fails
without my patch applied, and so is a separate issue in the Ambari Yarn stacks.
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message