spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dav...@apache.org
Subject spark git commit: [SPARK-6931] [PYSPARK] Cast Python time float values to int before serialization
Date Thu, 10 Sep 2015 18:20:18 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.2 7029cd12b -> 4862a80d2


[SPARK-6931] [PYSPARK] Cast Python time float values to int before serialization

Python time values return a floating point value, need to cast to integer before serialize
with struct.pack('!q', value)

https://issues.apache.org/jira/browse/SPARK-6931

Author: Bryan Cutler <bjcutler@us.ibm.com>

Closes #8594 from BryanCutler/py-write_long-backport-6931-1.2.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4862a80d
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/4862a80d
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/4862a80d

Branch: refs/heads/branch-1.2
Commit: 4862a80d2f992e41d30aa5121882f3452d8216b8
Parents: 7029cd1
Author: Bryan Cutler <bjcutler@us.ibm.com>
Authored: Thu Sep 10 11:20:02 2015 -0700
Committer: Davies Liu <davies.liu@gmail.com>
Committed: Thu Sep 10 11:20:02 2015 -0700

----------------------------------------------------------------------
 python/pyspark/worker.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/4862a80d/python/pyspark/worker.py
----------------------------------------------------------------------
diff --git a/python/pyspark/worker.py b/python/pyspark/worker.py
index 7e5343c..0aefd1f 100644
--- a/python/pyspark/worker.py
+++ b/python/pyspark/worker.py
@@ -39,9 +39,9 @@ utf8_deserializer = UTF8Deserializer()
 
 def report_times(outfile, boot, init, finish):
     write_int(SpecialLengths.TIMING_DATA, outfile)
-    write_long(1000 * boot, outfile)
-    write_long(1000 * init, outfile)
-    write_long(1000 * finish, outfile)
+    write_long(int(1000 * boot), outfile)
+    write_long(int(1000 * init), outfile)
+    write_long(int(1000 * finish), outfile)
 
 
 def add_path(path):


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message