drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5457) Support Spill to Disk for the Hash Aggregate Operator
Date Thu, 01 Jun 2017 22:58:04 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033872#comment-16033872
] 

ASF GitHub Bot commented on DRILL-5457:
---------------------------------------

Github user Ben-Zvi commented on a diff in the pull request:

    https://github.com/apache/drill/pull/822#discussion_r119749313
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java
---
    @@ -323,38 +717,32 @@ public AggOutcome doWork() {
                     if (EXTRA_DEBUG_1) {
                       logger.debug("Received new schema.  Batch has {} records.", incoming.getRecordCount());
                     }
    -//                newSchema = true;
                     this.cleanup();
                     // TODO: new schema case needs to be handled appropriately
                     return AggOutcome.UPDATE_AGGREGATOR;
     
                   case OK:
                     resetIndex();
    -                if (incoming.getRecordCount() == 0) {
    -                  continue;
    -                } else {
    -                  checkGroupAndAggrValues(currentIndex);
    -                  incIndex();
    -
    -                  if (EXTRA_DEBUG_1) {
    -                    logger.debug("Continuing outside loop");
    -                  }
    -                  continue outside;
    +
    +                if (EXTRA_DEBUG_1) {
    +                  logger.debug("Continuing outside loop");
                     }
    +                continue outside;
     
                   case NONE:
    -                // outcome = out;
    +                underlyingIndex = 0; // in case need to handle a spilled partition
    +                try { currentIndex = getVectorIndex(underlyingIndex); }
    +                catch (SchemaChangeException sc) { throw new DrillRuntimeException(sc);}
    --- End diff --
    
    Done -- all SCE catchers either call UnsupportedOperationException, or (when clearly not
possible then) IllegalStateException 



> Support Spill to Disk for the Hash Aggregate Operator
> -----------------------------------------------------
>
>                 Key: DRILL-5457
>                 URL: https://issues.apache.org/jira/browse/DRILL-5457
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Relational Operators
>    Affects Versions: 1.10.0
>            Reporter: Boaz Ben-Zvi
>            Assignee: Boaz Ben-Zvi
>             Fix For: 1.11.0
>
>
> Support gradual spilling memory to disk as the available memory gets too small to allow
in memory work for the Hash Aggregate Operator.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message