drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4715) Java compilation error for a query with large number of expressions
Date Wed, 22 Jun 2016 20:54:16 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345148#comment-15345148
] 

ASF GitHub Bot commented on DRILL-4715:
---------------------------------------

Github user sudheeshkatkam commented on a diff in the pull request:

    https://github.com/apache/drill/pull/521#discussion_r68133092
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/ClassGenerator.java
---
    @@ -215,22 +218,47 @@ public JVar declareVectorValueSetupAndMember(DirectExpression batchName,
TypedFi
         return vv;
       }
     
    +  public enum BlkCreateMode {
    +    TRUE,  // Create new block
    +    FALSE, // Do not create block; put into existing block.
    +    TRUE_IF_BOUND // Create new block only if # of expressions added hit upper-bound
(ExecConstants.CODE_GEN_EXP_IN_METHOD_SIZE)
    +  }
    +
       public HoldingContainer addExpr(LogicalExpression ex) {
    -    return addExpr(ex, true);
    +    // default behavior is always to put expression into new block.
    +    return addExpr(ex, BlkCreateMode.TRUE);
       }
     
    -  public HoldingContainer addExpr(LogicalExpression ex, boolean rotate) {
    -//    logger.debug("Adding next write {}", ex);
    -    if (rotate) {
    -      rotateBlock();
    +  public HoldingContainer addExpr(LogicalExpression ex, BlkCreateMode mode) {
    +    if (mode == BlkCreateMode.TRUE || mode == BlkCreateMode.TRUE_IF_BOUND) {
    +      rotateBlock(mode);
         }
    +
    +    for (LinkedList<SizedJBlock> b : blocks) {
    +      b.getLast().incCounter();
    +    }
    +
         return evaluationVisitor.addExpr(ex, this);
       }
     
       public void rotateBlock() {
    -    evaluationVisitor.previousExpressions.clear();
    -    for (LinkedList<JBlock> b : blocks) {
    -      b.add(new JBlock(true, true));
    +    // default behavior is always to create new block.
    +    rotateBlock(BlkCreateMode.TRUE);
    +  }
    +
    +  private void rotateBlock(BlkCreateMode mode) {
    +    boolean blockRotated = false;
    +    for (LinkedList<SizedJBlock> b : blocks) {
    +      if (mode == BlkCreateMode.TRUE ||
    +          (mode == BlkCreateMode.TRUE_IF_BOUND &&
    +            optionManager != null &&
    +            b.getLast().getCount() > optionManager.getOption(ExecConstants.CODE_GEN_EXP_IN_METHOD_SIZE).num_val))
{
    --- End diff --
    
    Drop `.num_val`. The OptionValidator is typed (LongValidator).


> Java compilation error for a query with large number of expressions
> -------------------------------------------------------------------
>
>                 Key: DRILL-4715
>                 URL: https://issues.apache.org/jira/browse/DRILL-4715
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Codegen
>            Reporter: Jinfeng Ni
>
> The following query would hit compilation error, when Drill generates and compiles the
run-time code. 
> Q1 :
> {code}
> select  expr1, expr2, expr3, ...., exprN
> from table
> {code} 
> In Q1, expr1, expr2, ..., exprN are non-trivial expression (in stead of simply column
reference), and N is big enough, then the run-time generated code may have a method which
goes beyond the 64k limit imposed by JVM. 
> This seems to be a regression from DRILL-3912 (Common subexpression elimination). CSE
tries to put as many expressions in one block as possible, to detect and eliminate as many
CSE as possible. However, this implies we may end up with big method with compilation error.
>     



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message