drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chunhui Shi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4777) Fuse generated code to reduce code size and gain performance improvement
Date Wed, 13 Jul 2016 07:45:20 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15374543#comment-15374543
] 

Chunhui Shi commented on DRILL-4777:
------------------------------------

Example code pieces for function compare_to_nulls_high and hash32 embedded in generated source
HashTableGenNN.java:

3021:                   //---- start of eval portion of compare_to_nulls_high function. ----//
3022:                   IntHolder out17 = new IntHolder();
3023:                   {
3024:                       final IntHolder out = new IntHolder();
3025:                       NullableBigIntHolder left = out12;
3026:                       NullableBigIntHolder right = out16;
3027:                        
3028:   GCompareBigIntVsBigInt$GCompareNullableBigIntVsNullableBigIntNullHigh_eval: {
3029:       outside:
3030:       {
3031:           if (left.isSet == 0) {
3032:               if (right.isSet == 0) {
3033:                   out.value = 0;
3034:                   break outside;
3035:               } else
3036:               {
3037:                   out.value = 1;
3038:                   break outside;
3039:               }
3040:           } else
3041:           if (right.isSet == 0) {
3042:               out.value = -1;
3043:               break outside;
3044:           }
3045:           out.value = left.value < right.value ? -1 : (left.value == right.value
? 0 : 1);
3046:       }
3047:   }
3048:    
3049:                       out17 = out;
3050:                   }



462:                IntHolder out127 = new IntHolder();
463:                {
464:                    final IntHolder out = new IntHolder();
465:                    NullableVarCharHolder in = out91;
466:                    IntHolder seed = out126;
467:                     
468:    Hash32FunctionsWithSeed$NullableVarCharHash_eval: {
469:        if (in.isSet == 0) {
470:            out.value = seed.value;
471:        } else
472:        {
473:            out.value = org.apache.drill.exec.expr.fn.impl.XXHash.hash32(in.start, in.end,
in.buffer, seed.value);
474:        }
475:    }
476:     
477:                    out127 = out;
478:                }
479:                //---- end of eval portion of hash32 function. ----//

> Fuse generated code to reduce code size and gain performance improvement
> ------------------------------------------------------------------------
>
>                 Key: DRILL-4777
>                 URL: https://issues.apache.org/jira/browse/DRILL-4777
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Chunhui Shi
>
> Drill generates code for operators, compile the classes and load them on the fly of a
query. However, in large query cases, the generated code will become hundreds KB or could
be even bigger. We have seen multiple issues reported when generated code is too big either
due to Java's size limit to one method or degrated performance of compiling or executing.
Also when I looked at JIT optimization logs, there are many complaining about 'hot method
too big'
> Some measures can be considered to reduce the code size, such as, 
> 1) For now Drill embed function calls' code directly into generated code, this make one
line function call to become 5-10 lines code in generated java classes. If we inject these
functions as a private functions of the classes and directly call them in main function body,
this could reduce code size while the cost of function call can be erased by JIT inline optimization.
> 2) Drill generate one variable for each column, if the column number became dozens to
one hundred, the code will be redundant, we could consider using an array to store the value
vectors and loop on top of it so the code size will be reduced even more.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message