Mailing-List: contact issues-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@impala.incubator.apache.org
Date: Wed, 12 Apr 2017 15:05:41 +0000 (UTC)
From: "Tim Armstrong (JIRA)" <jira@apache.org>
To: issues@impala.incubator.apache.org
Message-ID: <JIRA.13063522.1492009493000.266290.1492009541736@Atlassian.JIRA>
In-Reply-To: <JIRA.13063522.1492009493000@Atlassian.JIRA>
References: <JIRA.13063522.1492009493000@Atlassian.JIRA> <JIRA.13063522.1492009493542@jira-lw-us.apache.org>
Subject: [jira] [Created] (IMPALA-5200) Profile timers not updated during
 long-running sort
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 12 Apr 2017 15:05:47 -0000

Tim Armstrong created IMPALA-5200:
-------------------------------------

             Summary: Profile timers not updated during long-running sort
                 Key: IMPALA-5200
                 URL: https://issues.apache.org/jira/browse/IMPALA-5200
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend
    Affects Versions: Impala 2.9.0
            Reporter: Tim Armstrong
            Priority: Minor


If you have a query plan with a long-running sort operation (e.g. minutes), the profile timers are not updated to reflect the time spent in the sort until the sort starts returning rows.

E.g. this is a summary from a sort query that was running for a few hours (!). The summary was misleading and the "heat map" plan in the debug web UI is misleading - it showed the join as the "hot" operator. It would be ideal if we could somehow at least periodically update the time spent in the operator.
{code}
Operator              #Hosts   Avg Time   Max Time    #Rows  Est. #Rows   Peak Mem  Est. Peak Mem  Detail                   
----------------------------------------------------------------------------------------------------------------------------
05:MERGING-EXCHANGE        1    0.000ns    0.000ns        0     635.58M          0        -1.00 B  UNPARTITIONED            
03:SORT                    1    0.000ns    0.000ns        0     635.58M   47.86 GB      800.00 MB                           
02:HASH JOIN               1    4s859ms    4s859ms  771.02M     635.58M  162.11 MB       16.03 MB  INNER JOIN, BROADCAST    
|--04:EXCHANGE             1   38.988ms   38.988ms  247.20K     247.20K          0              0  BROADCAST                
|  01:SCAN HDFS            1    8s089ms    8s089ms  247.20K     247.20K    3.79 MB       32.00 MB  product b     
00:SCAN HDFS               1  209.997ms  209.997ms   15.09M     635.58M  185.27 MB      176.00 MB  sales a
{code}

http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Impala-join-query-running-slow


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)