spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dav...@apache.org
Subject spark git commit: [SPARK-4561] [PYSPARK] [SQL] turn Row into dict recursively
Date Sat, 08 Aug 2015 15:36:17 GMT
Repository: spark
Updated Branches:
  refs/heads/master 106c0789d -> 74a6541aa


[SPARK-4561] [PYSPARK] [SQL] turn Row into dict recursively

Add an option `recursive` to `Row.asDict()`, when True (default is False), it will convert
the nested Row into dict.

Author: Davies Liu <davies@databricks.com>

Closes #8006 from davies/as_dict and squashes the following commits:

922cc5a [Davies Liu] turn Row into dict recursively


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/74a6541a
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/74a6541a
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/74a6541a

Branch: refs/heads/master
Commit: 74a6541aa82bcd7a052b2e57b5ca55b7c316495b
Parents: 106c078
Author: Davies Liu <davies@databricks.com>
Authored: Sat Aug 8 08:36:14 2015 -0700
Committer: Davies Liu <davies.liu@gmail.com>
Committed: Sat Aug 8 08:36:14 2015 -0700

----------------------------------------------------------------------
 python/pyspark/sql/types.py | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/74a6541a/python/pyspark/sql/types.py
----------------------------------------------------------------------
diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py
index 6f74b71..e2e6f03 100644
--- a/python/pyspark/sql/types.py
+++ b/python/pyspark/sql/types.py
@@ -1197,13 +1197,36 @@ class Row(tuple):
         else:
             raise ValueError("No args or kwargs")
 
-    def asDict(self):
+    def asDict(self, recursive=False):
         """
         Return as an dict
+
+        :param recursive: turns the nested Row as dict (default: False).
+
+        >>> Row(name="Alice", age=11).asDict() == {'name': 'Alice', 'age': 11}
+        True
+        >>> row = Row(key=1, value=Row(name='a', age=2))
+        >>> row.asDict() == {'key': 1, 'value': Row(age=2, name='a')}
+        True
+        >>> row.asDict(True) == {'key': 1, 'value': {'name': 'a', 'age': 2}}
+        True
         """
         if not hasattr(self, "__fields__"):
             raise TypeError("Cannot convert a Row class into dict")
-        return dict(zip(self.__fields__, self))
+
+        if recursive:
+            def conv(obj):
+                if isinstance(obj, Row):
+                    return obj.asDict(True)
+                elif isinstance(obj, list):
+                    return [conv(o) for o in obj]
+                elif isinstance(obj, dict):
+                    return dict((k, conv(v)) for k, v in obj.items())
+                else:
+                    return obj
+            return dict(zip(self.__fields__, (conv(o) for o in self)))
+        else:
+            return dict(zip(self.__fields__, self))
 
     # let object acts like class
     def __call__(self, *args):


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message