spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From viirya <...@git.apache.org>
Subject [GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...
Date Mon, 20 Aug 2018 01:10:19 GMT
Github user viirya commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21859#discussion_r211122674
  
    --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
    @@ -1207,6 +1207,13 @@ object SQLConf {
           .intConf
           .createWithDefault(100)
     
    +  val RANGE_EXCHANGE_SAMPLE_CACHE_ENABLE =
    +    buildConf("spark.sql.execution.rangeExchange.sampleCache.enabled")
    +      .doc("When true, if the sampled data in range partitioning covers the whole data,
" +
    +        "the sampled data will be cached for the next sort.")
    +      .booleanConf
    +      .createWithDefault(true)
    --- End diff --
    
    Should it be an internal config?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message