ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 34618: Spark History Server in HDP 2.3 no longer needs tez.tar.gz, which now needs to be uploaded to HDFS during Tez Service Check
Date Mon, 25 May 2015 08:31:33 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34618/
-----------------------------------------------------------

(Updated May 25, 2015, 8:31 a.m.)


Review request for Ambari, Dmitro Lisnichenko, Jonathan Hurley, and Nate Cole.


Changes
-------

Addressed comments, ready to commit.


Bugs: AMBARI-11357
    https://issues.apache.org/jira/browse/AMBARI-11357


Repository: ambari


Description
-------

In HDP 2.3, Spark no longer needs Tez as a dependency.
This means that
1. Spark metainfo.xml does not need to co-host Tez-client
2. Spark History Server restart or RU restart does not need to copy tez tarball to HDFS. Note,
this is still needed in HDP 2.2
3. Tez needs to be responsible for copying its own tarball to HDFS because its service check
depends on it.


Diffs (updated)
-----

  ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/job_history_server.py
106b8cc 
  ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py
834e208 
  ambari-server/src/main/resources/common-services/SPARK/1.3.1.2.3/metainfo.xml PRE-CREATION

  ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/service_check.py
98973ae 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/SPARK/metainfo.xml 205ccc9 
  ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py dbefa5c 

Diff: https://reviews.apache.org/r/34618/diff/


Testing
-------

----------------------------------------------------------------------
Total run:742
Total errors:0
Total failures:0
OK

Copied changes files to a cluster with HDP 2.3.0.0 and HDFS, YARN, MR, ZK.
Then installed Spark, which did not require Tez, as expected.
Then installed Tez, whose service checked passed.
Then ran an RU and everything passed


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message