arrow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From w...@apache.org
Subject [arrow] branch master updated: ARROW-1268: [SITE][FOLLOWUP] Update Spark Post to Reflect Conf Change
Date Tue, 21 Nov 2017 23:24:53 GMT
This is an automated email from the ASF dual-hosted git repository.

wesm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git


The following commit(s) were added to refs/heads/master by this push:
     new 3fb1491  ARROW-1268: [SITE][FOLLOWUP] Update Spark Post to Reflect Conf Change
3fb1491 is described below

commit 3fb14914abfe89797f18a6cc5f62ec5cbe0ab2be
Author: Bryan Cutler <cutlerb@gmail.com>
AuthorDate: Tue Nov 21 18:24:49 2017 -0500

    ARROW-1268: [SITE][FOLLOWUP] Update Spark Post to Reflect Conf Change
    
    The Spark conf to enable arrow has changed from "spark.sql.execution.arrow.enable" to
"spark.sql.execution.arrow.enabled"
    
    Author: Bryan Cutler <cutlerb@gmail.com>
    
    Closes #1344 from BryanCutler/site-Spark-blog-conf-update-ARROW-1268 and squashes the
following commits:
    
    5e784351 [Bryan Cutler] update spark conf to enable arrow
---
 site/_posts/2017-07-26-spark-arrow.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/site/_posts/2017-07-26-spark-arrow.md b/site/_posts/2017-07-26-spark-arrow.md
index c4b16c0..211e5a4 100644
--- a/site/_posts/2017-07-26-spark-arrow.md
+++ b/site/_posts/2017-07-26-spark-arrow.md
@@ -57,7 +57,7 @@ the conversion to Arrow data can be done on the JVM and pushed back for
the Spar
 executors to perform in parallel, drastically reducing the load on the driver.
 
 As of the merging of [SPARK-13534][5], the use of Arrow when calling `toPandas()`
-needs to be enabled by setting the SQLConf "spark.sql.execution.arrow.enable" to
+needs to be enabled by setting the SQLConf "spark.sql.execution.arrow.enabled" to
 "true".  Let's look at a simple usage example.
 
 ```
@@ -84,7 +84,7 @@ In [2]: %time pdf = df.toPandas()
 CPU times: user 17.4 s, sys: 792 ms, total: 18.1 s
 Wall time: 20.7 s
 
-In [3]: spark.conf.set("spark.sql.execution.arrow.enable", "true")
+In [3]: spark.conf.set("spark.sql.execution.arrow.enabled", "true")
 
 In [4]: %time pdf = df.toPandas()
 CPU times: user 40 ms, sys: 32 ms, total: 72 ms                                 
@@ -118,7 +118,7 @@ It is planned to add pyarrow as a pyspark dependency so that
 
 Currently, the controlling SQLConf is disabled by default. This can be enabled
 programmatically as in the example above or by adding the line
-"spark.sql.execution.arrow.enable=true" to `SPARK_HOME/conf/spark-defaults.conf`.
+"spark.sql.execution.arrow.enabled=true" to `SPARK_HOME/conf/spark-defaults.conf`.
 
 Also, not all Spark data types are currently supported and limited to primitive
 types. Expanded type support is in the works and expected to also be in the Spark

-- 
To stop receiving notification emails like this one, please contact
['"commits@arrow.apache.org" <commits@arrow.apache.org>'].

Mime
View raw message