spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From a...@apache.org
Subject git commit: doctest updated for mapValues, flatMapValues in rdd.py
Date Sat, 22 Feb 2014 23:10:42 GMT
Repository: incubator-spark
Updated Branches:
  refs/heads/master 3ff077d48 -> 722199fab


doctest updated for mapValues, flatMapValues in rdd.py

Updated doctests for mapValues and flatMapValues in rdd.py

Author: jyotiska <jyotiska123@gmail.com>

Closes #621 from jyotiska/python_spark and squashes the following commits:

716f7cd [jyotiska] doctest updated for mapValues, flatMapValues in rdd.py


Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/722199fa
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/722199fa
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/722199fa

Branch: refs/heads/master
Commit: 722199fab072b4c19a82031c52e5d44f300bd2ea
Parents: 3ff077d
Author: jyotiska <jyotiska123@gmail.com>
Authored: Sat Feb 22 15:10:31 2014 -0800
Committer: Aaron Davidson <aaron@databricks.com>
Committed: Sat Feb 22 15:10:31 2014 -0800

----------------------------------------------------------------------
 python/pyspark/rdd.py | 10 ++++++++++
 1 file changed, 10 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/722199fa/python/pyspark/rdd.py
----------------------------------------------------------------------
diff --git a/python/pyspark/rdd.py b/python/pyspark/rdd.py
index 90f93a1..1330e61 100644
--- a/python/pyspark/rdd.py
+++ b/python/pyspark/rdd.py
@@ -946,6 +946,11 @@ class RDD(object):
         Pass each value in the key-value pair RDD through a flatMap function
         without changing the keys; this also retains the original RDD's
         partitioning.
+
+        >>> x = sc.parallelize([("a", ["x", "y", "z"]), ("b", ["p", "r"])])
+        >>> def f(x): return x
+        >>> x.flatMapValues(f).collect()
+        [('a', 'x'), ('a', 'y'), ('a', 'z'), ('b', 'p'), ('b', 'r')]
         """
         flat_map_fn = lambda (k, v): ((k, x) for x in f(v))
         return self.flatMap(flat_map_fn, preservesPartitioning=True)
@@ -955,6 +960,11 @@ class RDD(object):
         Pass each value in the key-value pair RDD through a map function
         without changing the keys; this also retains the original RDD's
         partitioning.
+
+        >>> x = sc.parallelize([("a", ["apple", "banana", "lemon"]), ("b", ["grapes"])])
+        >>> def f(x): return len(x)
+        >>> x.mapValues(f).collect()
+        [('a', 3), ('b', 1)]
         """
         map_values_fn = lambda (k, v): (k, f(v))
         return self.map(map_values_fn, preservesPartitioning=True)


Mime
View raw message