Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Thu, 1 Jun 2017 22:58:04 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.13068316.1493697798000.345999.1496357884713@Atlassian.JIRA>
In-Reply-To: <JIRA.13068316.1493697798000@Atlassian.JIRA>
References: <JIRA.13068316.1493697798000@Atlassian.JIRA> <JIRA.13068316.1493697798691@jira-lw-us.apache.org>
Subject: [jira] [Commented] (DRILL-5457) Support Spill to Disk for the Hash
 Aggregate Operator
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 01 Jun 2017 22:58:09 -0000


    [ https://issues.apache.org/jira/browse/DRILL-5457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033872#comment-16033872 ] 

ASF GitHub Bot commented on DRILL-5457:
---------------------------------------

Github user Ben-Zvi commented on a diff in the pull request:

    https://github.com/apache/drill/pull/822#discussion_r119749313
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java ---
    @@ -323,38 +717,32 @@ public AggOutcome doWork() {
                     if (EXTRA_DEBUG_1) {
                       logger.debug("Received new schema.  Batch has {} records.", incoming.getRecordCount());
                     }
    -//                newSchema = true;
                     this.cleanup();
                     // TODO: new schema case needs to be handled appropriately
                     return AggOutcome.UPDATE_AGGREGATOR;
     
                   case OK:
                     resetIndex();
    -                if (incoming.getRecordCount() == 0) {
    -                  continue;
    -                } else {
    -                  checkGroupAndAggrValues(currentIndex);
    -                  incIndex();
    -
    -                  if (EXTRA_DEBUG_1) {
    -                    logger.debug("Continuing outside loop");
    -                  }
    -                  continue outside;
    +
    +                if (EXTRA_DEBUG_1) {
    +                  logger.debug("Continuing outside loop");
                     }
    +                continue outside;
     
                   case NONE:
    -                // outcome = out;
    +                underlyingIndex = 0; // in case need to handle a spilled partition
    +                try { currentIndex = getVectorIndex(underlyingIndex); }
    +                catch (SchemaChangeException sc) { throw new DrillRuntimeException(sc);}
    --- End diff --
    
    Done -- all SCE catchers either call UnsupportedOperationException, or (when clearly not possible then) IllegalStateException 


> Support Spill to Disk for the Hash Aggregate Operator
> -----------------------------------------------------
>
>                 Key: DRILL-5457
>                 URL: https://issues.apache.org/jira/browse/DRILL-5457
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Relational Operators
>    Affects Versions: 1.10.0
>            Reporter: Boaz Ben-Zvi
>            Assignee: Boaz Ben-Zvi
>             Fix For: 1.11.0
>
>
> Support gradual spilling memory to disk as the available memory gets too small to allow in memory work for the Hash Aggregate Operator.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)