ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Zhang" <zjf...@apache.org>
Subject Re: Review Request 42051: [AMBARI-14561] Configuration changes of spark thrift server for Spark 1.6
Date Fri, 08 Jan 2016 06:11:32 GMT


> On Jan. 8, 2016, 6:07 a.m., Sumit Mohanty wrote:
> > Ship It!

Could you please hold on ? Actually I havn't resolved the follow thing, could you help on
that ?
The weird thing is that the configuration in HDP 2.3's spark-thrift-sparkconf.xml will be
appended to HDP-2.4's spark-thrift-sparkconf.xml. What I want is remove some configuration
in 2.3. And actually I make Spark 1.6 of HDP 2.4 extend Spark 1.2 in common-services, not
sure why it still use spark-thrift-sparkconf.xml in HDP 2.3


- Jeff


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42051/#review113419
-----------------------------------------------------------


On Jan. 8, 2016, 1:18 a.m., Jeff Zhang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42051/
> -----------------------------------------------------------
> 
> (Updated Jan. 8, 2016, 1:18 a.m.)
> 
> 
> Review request for Ambari and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-14561
>     https://issues.apache.org/jira/browse/AMBARI-14561
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> This is for HDP stack, in HDP stack, we'd like to do following changes for spark thrift
server
> - Remove advanced properties except queue name
> - Use yarn-client mode
> - Enable Dynamic Resource Allocation
> - Enable Fair scheduling policy
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/setup_spark.py
4b38572 
>   ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/templates/spark-thrift-fairscheduler.xml.j2
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-hive-site-override.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/configuration/spark-thrift-sparkconf.xml
PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/metainfo.xml 6c0e393

>   ambari-server/src/main/resources/stacks/HDP/2.4/services/SPARK/quicklinks/quicklinks.json
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/42051/diff/
> 
> 
> Testing
> -------
> 
> This patch doesn't solve the issue completly. The weird thing is that the configuration
in HDP 2.3's spark-thrift-sparkconf.xml will be appended to HDP-2.4's spark-thrift-sparkconf.xml.
And what I want is remove some configuration in 2.3. And actually I make Spark 1.6 of HDP
2.4 extend Spark 1.2 in common-services, not sure why it still use spark-thrift-sparkconf.xml
in HDP 2.3
> 
> 
> Thanks,
> 
> Jeff Zhang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message