spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takeshi Yamamuro (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-24498) Add JDK compiler for runtime codegen
Date Fri, 13 Jul 2018 12:58:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543025#comment-16543025
] 

Takeshi Yamamuro edited comment on SPARK-24498 at 7/13/18 12:57 PM:
--------------------------------------------------------------------

I found some weird gen'd code that jdk java compilers can't compile, e.g.,
 - gen'd code includes checked exceptions that are not caught inside
 [https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-8f85177fc023b09d1b6561e4c398275eR21]
 - cast errors
 [https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-258b71121d8d168e4d53cb5b6dc53ffeR893]
 - illegal import formats
 [https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-53c8a01004edfc2aabfe81570af05bb0R191]
 - illegal final qualifiers
 [https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-9853dcf5ce3d2ac1e94d473197ff5768L2669]
 - illegal generic arrays
 [https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-ead8a43fb83df762b597637f413a40a5R322]
 ...

Whether this ticket resolved or not, I think we'd be better to brush up gen'd codes so that
both jdk java compilers and janino could compile them. If no problem, how about filing a separate
jira for that and fixing them first? cc: [~smilegator] [~viirya] [~mgaido] [~kiszk]  [~rednaxelafx]


was (Author: maropu):
I found some weird gen'd code that jdk java compilers can't compile, e.g.,
 - gen'd code includes checked exceptions that are not caught inside
https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-8f85177fc023b09d1b6561e4c398275eR21
- cast errors
https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-258b71121d8d168e4d53cb5b6dc53ffeR893
- illegal import formats
https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-53c8a01004edfc2aabfe81570af05bb0R191
- illegal final qualifiers
https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-9853dcf5ce3d2ac1e94d473197ff5768L2669
- illegal generic arrays
https://github.com/apache/spark/compare/master...maropu:JdkCompiler#diff-ead8a43fb83df762b597637f413a40a5R322
...

Whether this ticket resolved or not, I think we'd be better to brush up gen'd codes so that
both jdk java compilers and janino could compile them. If no problem, how about filing a separate
jira for that and fixing them first? cc: [~smilegator] [~viirya] [~mgaido] [~kiszk]

> Add JDK compiler for runtime codegen
> ------------------------------------
>
>                 Key: SPARK-24498
>                 URL: https://issues.apache.org/jira/browse/SPARK-24498
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 2.3.0
>            Reporter: Xiao Li
>            Priority: Major
>
> In some cases, JDK compiler can generate smaller bytecode and take less time in compilation
compared to Janino. However, in some cases, Janino is better. We should support both for our
runtime codegen. Janino will be still our default runtime codegen compiler. 
> See the related JIRAs in DRILL: 
> - https://issues.apache.org/jira/browse/DRILL-1155
> - https://issues.apache.org/jira/browse/DRILL-4778
> - https://issues.apache.org/jira/browse/DRILL-5696



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message