spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ma...@apache.org
Subject git commit: Add insertInto and saveAsTable to Python API.
Date Sat, 19 Apr 2014 22:09:07 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.0 bfb09c6b8 -> 6ab071982


Add insertInto and saveAsTable to Python API.

Author: Michael Armbrust <michael@databricks.com>

Closes #447 from marmbrus/pythonInsert and squashes the following commits:

c7ab692 [Michael Armbrust] Keep docstrings < 72 chars.
ff62870 [Michael Armbrust] Add insertInto and saveAsTable to Python API.

(cherry picked from commit 10d04213ffda9c64b7e39c7debc433be8ea343c7)
Signed-off-by: Matei Zaharia <matei@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6ab07198
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/6ab07198
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/6ab07198

Branch: refs/heads/branch-1.0
Commit: 6ab071982bd400536abe99e96b2e7fc4b8bd6b94
Parents: bfb09c6
Author: Michael Armbrust <michael@databricks.com>
Authored: Sat Apr 19 15:08:54 2014 -0700
Committer: Matei Zaharia <matei@databricks.com>
Committed: Sat Apr 19 15:09:01 2014 -0700

----------------------------------------------------------------------
 python/pyspark/sql.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/6ab07198/python/pyspark/sql.py
----------------------------------------------------------------------
diff --git a/python/pyspark/sql.py b/python/pyspark/sql.py
index 27753d5..114fa13 100644
--- a/python/pyspark/sql.py
+++ b/python/pyspark/sql.py
@@ -305,6 +305,19 @@ class SchemaRDD(RDD):
         """
         self._jschema_rdd.registerAsTable(name)
 
+    def insertInto(self, tableName, overwrite = False):
+        """
+        Inserts the contents of this SchemaRDD into the specified table,
+        optionally overwriting any existing data.
+        """
+        self._jschema_rdd.insertInto(tableName, overwrite)
+
+    def saveAsTable(self, tableName):
+        """
+        Creates a new table with the contents of this SchemaRDD.
+        """
+        self._jschema_rdd.saveAsTable(tableName)
+
     def _toPython(self):
         # We have to import the Row class explicitly, so that the reference Pickler has is
         # pyspark.sql.Row instead of __main__.Row


Mime
View raw message