spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "holdenk (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-15369) Investigate selectively using Jython for parts of PySpark
Date Tue, 17 May 2016 19:46:12 GMT
holdenk created SPARK-15369:
-------------------------------

             Summary: Investigate selectively using Jython for parts of PySpark
                 Key: SPARK-15369
                 URL: https://issues.apache.org/jira/browse/SPARK-15369
             Project: Spark
          Issue Type: Improvement
          Components: PySpark
            Reporter: holdenk
            Priority: Minor


Transfering data from the JVM to the Python executor can be a substantial bottleneck. While
JYthon is not suitable for all UDFs or map functions, it may be suitable for some simple ones.
We should investigate the option of using JYthon to accelerate these small functions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message