spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From m...@apache.org
Subject spark git commit: [SPARK-6720][MLLIB] PySpark MultivariateStatisticalSummary unit test for normL1...
Date Tue, 07 Apr 2015 21:36:59 GMT
Repository: spark
Updated Branches:
  refs/heads/master e6f08fb42 -> fc957dc78


[SPARK-6720][MLLIB] PySpark MultivariateStatisticalSummary unit test for normL1...

... and normL2.
Add test cases to insufficient unit test for `normL1` and `normL2`.

Ref: https://github.com/apache/spark/pull/5359

Author: lewuathe <lewuathe@me.com>

Closes #5374 from Lewuathe/SPARK-6720 and squashes the following commits:

5541b24 [lewuathe] More accurate tests
dc5718c [lewuathe] [SPARK-6720] PySpark MultivariateStatisticalSummary unit test for normL1
and normL2


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fc957dc7
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/fc957dc7
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/fc957dc7

Branch: refs/heads/master
Commit: fc957dc78138e72036dbbadc9a54f155d318c038
Parents: e6f08fb
Author: lewuathe <lewuathe@me.com>
Authored: Tue Apr 7 14:36:57 2015 -0700
Committer: Xiangrui Meng <meng@databricks.com>
Committed: Tue Apr 7 14:36:57 2015 -0700

----------------------------------------------------------------------
 python/pyspark/mllib/tests.py | 7 +++++++
 1 file changed, 7 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/fc957dc7/python/pyspark/mllib/tests.py
----------------------------------------------------------------------
diff --git a/python/pyspark/mllib/tests.py b/python/pyspark/mllib/tests.py
index 47dad7d..61ef398 100644
--- a/python/pyspark/mllib/tests.py
+++ b/python/pyspark/mllib/tests.py
@@ -363,6 +363,13 @@ class StatTests(PySparkTestCase):
         self.assertEqual(10, len(summary.normL1()))
         self.assertEqual(10, len(summary.normL2()))
 
+        data2 = self.sc.parallelize(xrange(10)).map(lambda x: Vectors.dense(x))
+        summary2 = Statistics.colStats(data2)
+        self.assertEqual(array([45.0]), summary2.normL1())
+        import math
+        expectedNormL2 = math.sqrt(sum(map(lambda x: x*x, xrange(10))))
+        self.assertTrue(math.fabs(summary2.normL2()[0] - expectedNormL2) < 1e-14)
+
 
 class VectorUDTTests(PySparkTestCase):
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message