ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Balamohan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-15521) Change Spark Thrift Server default configuration for spark.hadoop.cacheConf
Date Fri, 25 Mar 2016 00:39:25 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-15521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211217#comment-15211217
] 

Rajesh Balamohan commented on AMBARI-15521:
-------------------------------------------

[~zjffdu] - This is only for spark 1.6 in HDP 2.4.  

Setting "spark.hadoop.cacheConf=true" creates a JobConf that will be cached per RDD.  If job
involves 1000s of RDD, this could cache too many jobconfs which can adversely impact GC. Set
this parameter to false in such cases.

> Change Spark Thrift Server default configuration for spark.hadoop.cacheConf
> ---------------------------------------------------------------------------
>
>                 Key: AMBARI-15521
>                 URL: https://issues.apache.org/jira/browse/AMBARI-15521
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Bikas Saha
>             Fix For: 2.2.2
>
>         Attachments: AMBARI-15521.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message