ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sumit Mohanty" <smoha...@hortonworks.com>
Subject Re: Review Request 42051: [AMBARI-14561] Configuration changes of spark thrift server for Spark 1.6
Date Mon, 18 Jan 2016 19:31:03 GMT


> On Jan. 18, 2016, 1:58 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml,
line 70
> > <https://reviews.apache.org/r/42051/diff/4/?file=1199119#file1199119line70>
> >
> >     Ambari upgrade will automatically add these properties for already installed
cluster. Is that OK? can you test one upgrade scenario where spark is already installed using
an older version of Ambari?
> 
> Jeff Zhang wrote:
>     How can I do an ambari upgrade with this change ?

This is a non-issue.


- Sumit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42051/#review114925
-----------------------------------------------------------


On Jan. 16, 2016, 1:13 a.m., Jeff Zhang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42051/
> -----------------------------------------------------------
> 
> (Updated Jan. 16, 2016, 1:13 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Saisai Shao, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-14561
>     https://issues.apache.org/jira/browse/AMBARI-14561
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> This is for HDP stack, in HDP stack, we'd like to do following changes for spark thrift
server
> - Remove advanced properties except queue name
> - Use yarn-client mode
> - Enable Dynamic Resource Allocation
> - Enable Fair scheduling policy
> 
> 
> Diffs
> -----
> 
>   ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
c3ffc7b 
>   ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
68240bd 
>   ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_spark.py
4b38572 
>   ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/spark_service.py
d4c6732 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-defaults.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-fairscheduler.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 6c0e393

> 
> Diff: https://reviews.apache.org/r/42051/diff/
> 
> 
> Testing
> -------
> 
> This patch doesn't solve the issue completly. The weird thing is that the configuration
in HDP 2.3's spark-thrift-sparkconf.xml will be appended to HDP-2.4's spark-thrift-sparkconf.xml.
And what I want is remove some configuration in 2.3. And actually I make Spark 1.6 of HDP
2.4 extend Spark 1.2 in common-services, not sure why it still use spark-thrift-sparkconf.xml
in HDP 2.3
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message