spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject spark git commit: [SPARK-13482][MINOR][CONFIGURATION] Make consistency of the configuraiton named in TransportConf.
Date Thu, 25 Feb 2016 07:52:19 GMT
Repository: spark
Updated Branches:
  refs/heads/master 4d2864b2d -> 264533b55


[SPARK-13482][MINOR][CONFIGURATION] Make consistency of the configuraiton named in TransportConf.

`spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024 as integer
and the other one is '2m' as string.
"2m" is recommanded in document but it will go wrong if the code goes into `TransportConf#memoryMapBytes`.

[Jira](https://issues.apache.org/jira/browse/SPARK-13482)

Author: huangzhaowei <carlmartinmax@gmail.com>

Closes #11360 from SaintBacchus/SPARK-13482.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/264533b5
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/264533b5
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/264533b5

Branch: refs/heads/master
Commit: 264533b553be806b6c45457201952e83c028ec78
Parents: 4d2864b
Author: huangzhaowei <carlmartinmax@gmail.com>
Authored: Wed Feb 24 23:52:17 2016 -0800
Committer: Reynold Xin <rxin@databricks.com>
Committed: Wed Feb 24 23:52:17 2016 -0800

----------------------------------------------------------------------
 .../main/java/org/apache/spark/network/util/TransportConf.java    | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/264533b5/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java
----------------------------------------------------------------------
diff --git a/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java
b/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java
index 115135d..9f030da 100644
--- a/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java
+++ b/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java
@@ -132,7 +132,8 @@ public class TransportConf {
    * memory mapping has high overhead for blocks close to or below the page size of the OS.
    */
   public int memoryMapBytes() {
-    return conf.getInt("spark.storage.memoryMapThreshold", 2 * 1024 * 1024);
+    return Ints.checkedCast(JavaUtils.byteStringAsBytes(
+      conf.get("spark.storage.memoryMapThreshold", "2m")));
   }
 
   /**


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message