spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kazuaki Ishizaki (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-16213) Reduce runtime overhead of a program that creates an primitive array in DataFrame
Date Mon, 05 Sep 2016 12:33:20 GMT

     [ https://issues.apache.org/jira/browse/SPARK-16213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kazuaki Ishizaki updated SPARK-16213:
-------------------------------------
    Affects Version/s: 2.0.0

> Reduce runtime overhead of a program that creates an primitive array in DataFrame
> ---------------------------------------------------------------------------------
>
>                 Key: SPARK-16213
>                 URL: https://issues.apache.org/jira/browse/SPARK-16213
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.0.0
>            Reporter: Kazuaki Ishizaki
>
> Reduce runtime overhead of a program that creates an primitive array in DataFrame
> When a program creates an array in DataFrame, the code generator creates boxing operations.
If an array is for primitive type, there are some opportunities for optimizations in generated
code to reduce runtime overhead.
> Here is a simple example that has generated code with boxing operation
> {code}
> val df = sparkContext.parallelize(Seq(0.0d, 1.0d), 1).toDF
> df.selectExpr("Array(value + 1.1d, value + 2.2d)").show
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message