impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <>
Subject [jira] [Created] (IMPALA-5200) Profile timers not updated during long-running sort
Date Wed, 12 Apr 2017 15:05:41 GMT
Tim Armstrong created IMPALA-5200:

             Summary: Profile timers not updated during long-running sort
                 Key: IMPALA-5200
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend
    Affects Versions: Impala 2.9.0
            Reporter: Tim Armstrong
            Priority: Minor

If you have a query plan with a long-running sort operation (e.g. minutes), the profile timers
are not updated to reflect the time spent in the sort until the sort starts returning rows.

E.g. this is a summary from a sort query that was running for a few hours (!). The summary
was misleading and the "heat map" plan in the debug web UI is misleading - it showed the join
as the "hot" operator. It would be ideal if we could somehow at least periodically update
the time spent in the operator.
Operator              #Hosts   Avg Time   Max Time    #Rows  Est. #Rows   Peak Mem  Est. Peak
Mem  Detail                   
05:MERGING-EXCHANGE        1    0.000ns    0.000ns        0     635.58M          0       
-1.00 B  UNPARTITIONED            
03:SORT                    1    0.000ns    0.000ns        0     635.58M   47.86 GB      800.00
02:HASH JOIN               1    4s859ms    4s859ms  771.02M     635.58M  162.11 MB       16.03
|--04:EXCHANGE             1   38.988ms   38.988ms  247.20K     247.20K          0       
      0  BROADCAST                
|  01:SCAN HDFS            1    8s089ms    8s089ms  247.20K     247.20K    3.79 MB       32.00
MB  product b     
00:SCAN HDFS               1  209.997ms  209.997ms   15.09M     635.58M  185.27 MB      176.00
MB  sales a

This message was sent by Atlassian JIRA

View raw message