drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chunhui Shi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-4777) Fuse generated code to reduce code size and gain performance improvement
Date Wed, 13 Jul 2016 00:55:20 GMT
Chunhui Shi created DRILL-4777:

             Summary: Fuse generated code to reduce code size and gain performance improvement
                 Key: DRILL-4777
                 URL: https://issues.apache.org/jira/browse/DRILL-4777
             Project: Apache Drill
          Issue Type: Bug
            Reporter: Chunhui Shi

Drill generates code for operators, compile the classes and load them on the fly of a query.
However, in large query cases, the generated code will become hundreds KB or could be even
bigger. We have seen multiple issues reported when generated code is too big either due to
Java's size limit to one method or degrated performance of compiling or executing. Also when
I looked at JIT optimization logs, there are many complaining about 'hot method too big'

Some measures can be considered to reduce the code size, such as, 
1) For now Drill embed function calls' code directly into generated code, this make one line
function call to become 5-10 lines code in generated java classes. If we inject these functions
as a private functions of the classes and directly call them in main function body, this could
reduce code size while the cost of function call can be erased by JIT inline optimization.

2) Drill generate one variable for each column, if the column number became dozens to one
hundred, the code will be redundant, we could consider using an array to store the value vectors
and loop on top of it so the code size will be reduced even more.

This message was sent by Atlassian JIRA

View raw message