impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-841) Investigate perf issue with aggregations on hbase tables
Date Mon, 12 Jun 2017 21:10:01 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tim Armstrong resolved IMPALA-841.
----------------------------------
    Resolution: Cannot Reproduce

I doubt we have enough info to investigate at this point.

> Investigate perf issue with aggregations on hbase tables
> --------------------------------------------------------
>
>                 Key: IMPALA-841
>                 URL: https://issues.apache.org/jira/browse/IMPALA-841
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Perf Investigation
>    Affects Versions: Impala 1.2.3
>            Reporter: Nong Li
>            Priority: Minor
>
> We seem to have a perf issue for these queries. The profile indicates we are 
> spending a huge amount of time doing the aggregation but it's possible something
> is wrong how the profiles are measuring:
> In this case we are spending 110 seconds to aggregate 750K rows.
> More details: https://groups.google.com/a/cloudera.org/forum/#!topic/impala-user/KVCw5T4wC7s
> Averaged Fragment 1:(Active: 2m6s, % non-child: 0.00%)
>       split sizes:  min: 0.00 , max: 0.00 , avg: 0.00 , stddev: 0.00 
>       completion times: min:29s050ms  max:3m58s  mean: 2m6s  stddev:50s954ms
>       execution rates: min:0.00 /sec  max:0.00 /sec  mean:0.00 /sec  stddev:0.00 /sec
>       num instances: 25
>        - AverageThreadTokens: 1.00 
>        - PeakMemoryUsage: 1.04 MB
>        - RowsProduced: 1
>       CodeGen:(Active: 78.268ms, % non-child: 0.10%)
>          - CodegenTime: 882.862us
>          - CompileTime: 65.74ms
>          - LoadTime: 13.192ms
>          - ModuleFileSize: 69.84 KB
>       DataStreamSender (dst_id=2):(Active: 533.247us, % non-child: 0.00%)
>          - BytesSent: 16.00 B
>          - NetworkThroughput(*): 15.32 KB/sec
>          - OverallThroughput: 29.78 KB/sec
>          - SerializeBatchTime: 154.552us
>          - ThriftTransmitTime(*): 7.911ms
>          - UncompressedRowBatchSize: 16.00 B
>       AGGREGATION_NODE (id=1):(Active: 2m6s, % non-child: 83.85%)
>          - BuildBuckets: 1.02K (1024)
>          - BuildTime: 1m49s
>          - GetResultsTime: 132.109us
>          - LoadFactor: 0.00 
>          - MemoryUsed: 32.01 KB
>          - RowsReturned: 1
>          - RowsReturnedRate: 0
>       HBASE_SCAN_NODE (id=0):(Active: 17s264ms, % non-child: 16.15%)
>          - BytesRead: 31.83 MB
>          - HBaseTableScanner.ScanSetup: 388.495ms
>          - MemoryUsed: 0.00 
>          - RowsRead: 751.73K (751727)
>          - RowsReturned: 751.73K (751727)
>          - RowsReturnedRate: 44.38 K/sec
>          - ScannerThreadsInvoluntaryContextSwitches: 501
>          - ScannerThreadsTotalWallClockTime: 16s787ms
>            - MaterializeTupleTime(*): 630.329ms
>            - ScannerThreadsSysTime: 72.29ms
>            - ScannerThreadsUserTime: 2s847ms
>          - ScannerThreadsVoluntaryContextSwitches: 806
>          - TotalRawHBaseReadTime(*): 14s526ms
>          - TotalReadThroughput: 264.18 KB/sec



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message