impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeszy <jes...@gmail.com>
Subject Re: Estimate peak memory VS used peak memory
Date Fri, 23 Feb 2018 08:47:55 GMT
Hey Fawze,

Answers inline.

On 23 February 2018 at 01:23, Fawze Abujaber <fawzeaj@gmail.com> wrote:
> There is no option in the admission control to setup memory limit per query,
> the memory limit is per pool and there is a default memory per node for
> query.

per node for query memory limit multiplied by number of nodes gives
you a per query memory limit. I agree its confusing that the
configurations mix and match between per-node and aggregated values.
In this case there's a good reason though, as a single node running
out of memory will lead to query failure, meaning that in addition to
total memory used, distribution of memory usage between hosts also
matters.

> I have hundreds of impala queries and more add hoc queries, making a pool
> for each query is not a visible solution.
>
> still waiting to understand how the estimate per node related to the default
> memory per node I set up per pool, is it used in the decision of queuing and
> killing the query? and if this is true how it was not kill a query that was
> estimated it needs 8.2TB memory per node.
>
> Understanding on which parameters impala decides to kill a query can help
> understand to define and divide the memory between the pools.

If you set mem_limit at any level (service level, pool level, or query
level), it will be used for admission control purposes instead of
estimates. So a 8.2TB estimate would not be a problem, if impala can
reserve mem_limit amount on each host, it will start running the
query.

> Passing memory limit per query manually is also not visible and such
> settings not needs admission control.
>
> I have support pool that runs ad hoc query and I can not ask them to use
> memory limit per query, and I have analytics pool which is fully business
> and I can rely on admission control if it extremely in accurate.

It's a bit tricky to use memory-based admission control with
non-trivial ad hoc queries. For simple ad-hoc queries, you can try to
come up with a 'good enough' mem_limit, or omit mem_limit and trust
impala's estimations. You can check the estimated vs. actual values
for a representative set of ad hoc queries to see what would work in
your case. I've found that people tend to go with a large enough
mem_limit for the ad hoc pool.

> Can someone explain me exactly which recommended setting to use per pool and
> which of them rely on impala memory estimates?

The documentation of admission control
(https://impala.apache.org/docs/build/html/topics/impala_admission.html)
gives you a good view on how stuff works, but you will have to figure
out how to use these features for your specific use case. That said,
when using memory based admission control, it is best practice to
always use a mem_limit due to potential inaccuracy of estimates as
well as potential variance of estimates between Impala releases. Keep
in mind that you can opt to set a default mem_limit for one pool and
leave it unset for another.

> So my conclusion right now to avoid using any settings rely on the estimates
> and to ignore the estimates when I want to evaluate query.

Sounds good.

> @mostafa, since my issue with all the query, I think the profile will not
> help me to solve such huge issue.
>
> I’m planning to move a way from Vertica and rely on impala as a sql engine
> and now fully confused how I can do this if I can’t use the admission
> control.
>
> Last think, is it recommend to use the impala admission control?

Yes. Admission control can take a while to understand, but if done
right, it works.

HTH

> On Fri, 23 Feb 2018 at 1:56 Alexander Behm <alex.behm@cloudera.com> wrote:
>>
>> The planner memory estimates are conservative and sometimes extremely
>> inaccurate. In their current form, they are rarely appropriate for admission
>> decisions.
>>
>> The recommended practice for memory-based admission control it to set a
>> mem_limit for every query. You can make this easier by setting up different
>> pools with different mem_limits, e.g. a small/medium/big queries pool or
>> similar.
>>
>> On Thu, Feb 22, 2018 at 3:00 PM, Mostafa Mokhtar <mmokhtar@cloudera.com>
>> wrote:
>>>
>>> It is recommended to set a per query memory limit as part of admission
>>> and not rely on estimates as they are sometimes inaccurate.
>>> Can you please include the full query profile?
>>>
>>>
>>> On Thu, Feb 22, 2018 at 12:13 PM, Fawze Abujaber <fawzeaj@gmail.com>
>>> wrote:
>>>>
>>>> Hi Mostafa,
>>>>
>>>> It's not a specific query, almost all the query has such differene
>>>> between the 2 values.
>>>>
>>>> I can see even queries showing the estimate per node is 8.2 Tib
>>>>
>>>> User: psanalytics
>>>>
>>>> Database: default
>>>>
>>>> Query Type: QUERY
>>>> Coordinator: slpr-dhc014.lpdomain.com
>>>>
>>>> Duration: 6.48s
>>>>
>>>> Rows Produced: 708
>>>> Estimated per Node Peak Memory: 8.2 TiB
>>>>
>>>> Per Node Peak Memory Usage: 1.1 GiB
>>>>
>>>> Pool: root.impanalytics
>>>> Threads: CPU Time: 20.1m
>>>>
>>>>
>>>>
>>>> How you can explain this behavior, and for sure i don't have 8.2 Tib
>>>> memory per node to give neither you.
>>>>
>>>> Can you please explain me how i should treat Estimated per Node Peak
>>>> Memory and if it used by impala for the resource pool and admission control
>>>> and what is the relation of this value to the default memory per node that
i
>>>> setup for each resource pool?
>>>>
>>>> Below is part of one of the queries profile which the estimate per node
>>>> was ~ @GB and the used was 200MB per node.
>>>>
>>>>
>>>>  Instance 744de1b6228736fa:b54bfaa70000000f
>>>> (host=slpr-dhc004.lpdomain.com:22000):(Total: 1s455ms, non-child: 1s292ms,
%
>>>> non-child: 88.82%)
>>>>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> 6:1/1.20 MB 2:1/199.83 KB 1:1/1.20 MB 10:2/1.42 MB 8:1/225.57 KB 9:1/191.64
>>>> KB 5:2/289.57 KB 3:2/1012.83 KB
>>>>         MemoryUsage(500.000ms): 6.09 MB, 6.09 MB, 3.00 MB
>>>>         ThreadUsage(500.000ms): 1, 1, 1
>>>>          - AverageThreadTokens: 1.00
>>>>          - BloomFilterBytes: 0
>>>>          - PeakMemoryUsage: 7.17 MB (7521751)
>>>>          - PeakReservation: 0
>>>>          - PeakUsedReservation: 0
>>>>          - PerHostPeakMemUsage: 106.53 MB (111709581)
>>>>          - RowsProduced: 32.83K (32826)
>>>>          - TotalNetworkReceiveTime: 0.000ns
>>>>          - TotalNetworkSendTime: 1s297ms
>>>>          - TotalStorageWaitTime: 234.356ms
>>>>          - TotalThreadsInvoluntaryContextSwitches: 66 (66)
>>>>          - TotalThreadsTotalWallClockTime: 1s715ms
>>>>            - TotalThreadsSysTime: 5.998ms
>>>>            - TotalThreadsUserTime: 124.975ms
>>>>          - TotalThreadsVoluntaryContextSwitches: 303 (303)
>>>>         Fragment Instance Lifecycle Timings:
>>>>            - ExecTime: 1s394ms
>>>>              - ExecTreeExecTime: 67.115ms
>>>>            - OpenTime: 32.795ms
>>>>              - ExecTreeOpenTime: 73.243us
>>>>            - PrepareTime: 27.602ms
>>>>              - ExecTreePrepareTime: 243.141us
>>>>         DataStreamSender (dst_id=11):(Total: 38.747ms, non-child:
>>>> 38.747ms, % non-child: 100.00%)
>>>>            - BytesSent: 39.71 MB (41643000)
>>>>            - NetworkThroughput(*): 1.97 GB/sec
>>>>            - OverallThroughput: 1.00 GB/sec
>>>>            - PeakMemoryUsage: 59.38 KB (60800)
>>>>            - RowsReturned: 32.83K (32826)
>>>>            - SerializeBatchTime: 16.860ms
>>>>            - TransmitDataRPCTime: 19.698ms
>>>>            - UncompressedRowBatchSize: 77.58 MB (81350840)
>>>>         CodeGen:(Total: 56.573ms, non-child: 56.573ms, % non-child:
>>>> 100.00%)
>>>>            - CodegenTime: 1.299ms
>>>>            - CompileTime: 10.672ms
>>>>            - LoadTime: 0.000ns
>>>>            - ModuleBitcodeSize: 1.96 MB (2050180)
>>>>            - NumFunctions: 16 (16)
>>>>            - NumInstructions: 250 (250)
>>>>            - OptimizationTime: 21.023ms
>>>>            - PeakMemoryUsage: 125.00 KB (128000)
>>>>            - PrepareTime: 24.116ms
>>>>         SUBPLAN_NODE (id=6):(Total: 67.311ms, non-child: 12.013ms, %
>>>> non-child: 17.85%)
>>>>            - PeakMemoryUsage: 627.94 KB (643015)
>>>>            - RowsReturned: 32.77K (32768)
>>>>            - RowsReturnedRate: 486.81 K/sec
>>>>           NESTED_LOOP_JOIN_NODE (id=9):(Total: 33.999ms, non-child:
>>>> 25.197ms, % non-child: 74.11%)
>>>>              - BuildRows: 0 (0)
>>>>              - BuildTime: 0.000ns
>>>>              - PeakMemoryUsage: 24.00 KB (24576)
>>>>              - ProbeRows: 32.83K (32826)
>>>>              - ProbeTime: 0.000ns
>>>>              - RowsReturned: 16.80M (16795311)
>>>>              - RowsReturnedRate: 493.99 M/sec
>>>>             Nested Loop Join Builder:
>>>>                - PeakMemoryUsage: 8.00 KB (8192)
>>>>             SINGULAR_ROW_SRC_NODE (id=7):
>>>>                - PeakMemoryUsage: 0
>>>>                - RowsReturned: 0 (0)
>>>>                - RowsReturnedRate: 0
>>>>           UNNEST_NODE (id=8):(Total: 8.801ms, non-child: 8.801ms, %
>>>> non-child: 100.00%)
>>>>              - AvgCollectionSize: 1.00
>>>>              - MaxCollectionSize: 1 (1)
>>>>              - MinCollectionSize: 1 (1)
>>>>              - NumCollections: 32.83K (32826)
>>>>              - PeakMemoryUsage: 0
>>>>              - RowsReturned: 1 (1)
>>>>              - RowsReturnedRate: 113.00 /sec
>>>>         HDFS_SCAN_NODE (id=5):(Total: 21.299ms, non-child: 21.299ms, %
>>>> non-child: 100.00%)
>>>>           Hdfs split stats (<volume id>:<# splits>/<split
lengths>):
>>>> 6:1/1.20 MB 2:1/199.83 KB 1:1/1.20 MB 10:2/1.42 MB 8:1/225.57 KB 9:1/191.64
>>>> KB 5:2/289.57 KB 3:2/1012.83 KB
>>>>           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 11 out
>>>> of 11
>>>>           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0%
>>>> 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>>           File Formats: PARQUET/SNAPPY:66
>>>>           BytesRead(500.000ms): 2.26 MB, 2.26 MB, 2.26 MB
>>>>            - FooterProcessingTime: (Avg: 12.082ms ; Min: 9.079ms ; Max:
>>>> 29.275ms ; Number of samples: 11)
>>>>            - AverageHdfsReadThreadConcurrency: 0.00
>>>>            - AverageScannerThreadConcurrency: 0.00
>>>>            - BytesRead: 2.26 MB (2373738)
>>>>            - BytesReadDataNodeCache: 0
>>>>            - BytesReadLocal: 2.26 MB (2373738)
>>>>            - BytesReadRemoteUnexpected: 0
>>>>            - BytesReadShortCircuit: 2.26 MB (2373738)
>>>>            - CachedFileHandlesHitCount: 0 (0)
>>>>            - CachedFileHandlesMissCount: 77 (77)
>>>>            - DecompressionTime: 1.543ms
>>>>            - MaxCompressedTextFileLength: 0
>>>>            - NumColumns: 6 (6)
>>>>            - NumDictFilteredRowGroups: 0 (0)
>>>>            - NumDisksAccessed: 8 (8)
>>>>            - NumRowGroups: 11 (11)
>>>>            - NumScannerThreadsStarted: 11 (11)
>>>>            - NumScannersWithNoReads: 0 (0)
>>>>            - NumStatsFilteredRowGroups: 0 (0)
>>>>            - PeakMemoryUsage: 6.51 MB (6821203)
>>>>            - PerReadThreadRawHdfsThroughput: 764.51 MB/sec
>>>>            - RemoteScanRanges: 0 (0)
>>>>            - RowBatchQueueGetWaitTime: 18.825ms
>>>>            - RowBatchQueuePutWaitTime: 0.000ns
>>>>            - RowsRead: 73.46K (73464)
>>>>            - RowsReturned: 32.83K (32826)
>>>>            - RowsReturnedRate: 1.54 M/sec
>>>>            - ScanRangesComplete: 11 (11)
>>>>            - ScannerThreadsInvoluntaryContextSwitches: 1 (1)
>>>>            - ScannerThreadsTotalWallClockTime: 287.702ms
>>>>              - MaterializeTupleTime(*): 11.921ms
>>>>              - ScannerThreadsSysTime: 999.000us
>>>>              - ScannerThreadsUserTime: 26.990ms
>>>>            - ScannerThreadsVoluntaryContextSwitches: 232 (232)
>>>>            - TotalRawHdfsReadTime(*): 2.961ms
>>>>            - TotalReadThroughput: 1.51 MB/sec
>>>>       Instance 744de1b6228736fa:b54bfaa70000000d
>>>> (host=slpr-dhc002.lpdomain.com:22000):(Total: 1s453ms, non-child: 1s254ms,
%
>>>> non-child: 86.28%)
>>>>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> 0:1/1.32 MB 4:1/1.18 MB 5:1/1.19 MB 10:2/1.06 MB
>>>>         MemoryUsage(500.000ms): 5.32 MB, 5.32 MB, 1.27 MB
>>>>         ThreadUsage(500.000ms): 1, 1, 1
>>>>          - AverageThreadTokens: 1.00
>>>>          - BloomFilterBytes: 0
>>>>          - PeakMemoryUsage: 5.86 MB (6145964)
>>>>          - PeakReservation: 0
>>>>          - PeakUsedReservation: 0
>>>>          - PerHostPeakMemUsage: 106.66 MB (111842766)
>>>>          - RowsProduced: 30.34K (30340)
>>>>          - TotalNetworkReceiveTime: 0.000ns
>>>>          - TotalNetworkSendTime: 1s256ms
>>>>          - TotalStorageWaitTime: 106.530ms
>>>>          - TotalThreadsInvoluntaryContextSwitches: 177 (177)
>>>>          - TotalThreadsTotalWallClockTime: 1s557ms
>>>>            - TotalThreadsSysTime: 6.998ms
>>>>            - TotalThreadsUserTime: 143.976ms
>>>>          - TotalThreadsVoluntaryContextSwitches: 136 (136)
>>>>         Fragment Instance Lifecycle Timings:
>>>>            - ExecTime: 1s390ms
>>>>              - ExecTreeExecTime: 91.500ms
>>>>            - OpenTime: 34.085ms
>>>>              - ExecTreeOpenTime: 81.436us
>>>>            - PrepareTime: 28.524ms
>>>>              - ExecTreePrepareTime: 217.949us
>>>>         DataStreamSender (dst_id=11):(Total: 48.674ms, non-child:
>>>> 48.674ms, % non-child: 100.00%)
>>>>            - BytesSent: 36.61 MB (38388760)
>>>>            - NetworkThroughput(*): 1.51 GB/sec
>>>>            - OverallThroughput: 752.15 MB/sec
>>>>            - PeakMemoryUsage: 59.38 KB (60800)
>>>>            - RowsReturned: 30.34K (30340)
>>>>            - SerializeBatchTime: 23.132ms
>>>>            - TransmitDataRPCTime: 23.609ms
>>>>            - UncompressedRowBatchSize: 71.70 MB (75187520)
>>>>         CodeGen:(Total: 59.024ms, non-child: 59.024ms, % non-child:
>>>> 100.00%)
>>>>            - CodegenTime: 1.382ms
>>>>            - CompileTime: 10.435ms
>>>>            - LoadTime: 0.000ns
>>>>            - ModuleBitcodeSize: 1.96 MB (2050180)
>>>>            - NumFunctions: 16 (16)
>>>>            - NumInstructions: 250 (250)
>>>>            - OptimizationTime: 22.325ms
>>>>            - PeakMemoryUsage: 125.00 KB (128000)
>>>>            - PrepareTime: 25.459ms
>>>>         SUBPLAN_NODE (id=6):(Total: 91.711ms, non-child: 20.997ms, %
>>>> non-child: 22.90%)
>>>>            - PeakMemoryUsage: 699.55 KB (716335)
>>>>            - RowsReturned: 29.70K (29696)
>>>>            - RowsReturnedRate: 323.80 K/sec
>>>>           NESTED_LOOP_JOIN_NODE (id=9):(Total: 53.723ms, non-child:
>>>> 40.263ms, % non-child: 74.95%)
>>>>              - BuildRows: 0 (0)
>>>>              - BuildTime: 0.000ns
>>>>              - PeakMemoryUsage: 24.00 KB (24576)
>>>>              - ProbeRows: 30.34K (30340)
>>>>              - ProbeTime: 0.000ns
>>>>              - RowsReturned: 15.43M (15426890)
>>>>              - RowsReturnedRate: 287.16 M/sec
>>>>             Nested Loop Join Builder:
>>>>                - PeakMemoryUsage: 8.00 KB (8192)
>>>>             SINGULAR_ROW_SRC_NODE (id=7):
>>>>                - PeakMemoryUsage: 0
>>>>                - RowsReturned: 0 (0)
>>>>                - RowsReturnedRate: 0
>>>>           UNNEST_NODE (id=8):(Total: 13.460ms, non-child: 13.460ms, %
>>>> non-child: 100.00%)
>>>>              - AvgCollectionSize: 1.00
>>>>              - MaxCollectionSize: 1 (1)
>>>>              - MinCollectionSize: 1 (1)
>>>>              - NumCollections: 30.34K (30340)
>>>>              - PeakMemoryUsage: 0
>>>>              - RowsReturned: 1 (1)
>>>>              - RowsReturnedRate: 74.00 /sec
>>>>         HDFS_SCAN_NODE (id=5):(Total: 16.990ms, non-child: 16.990ms, %
>>>> non-child: 100.00%)
>>>>           Hdfs split stats (<volume id>:<# splits>/<split
lengths>):
>>>> 0:1/1.32 MB 4:1/1.18 MB 5:1/1.19 MB 10:2/1.06 MB
>>>>           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 5 out of
>>>> 5
>>>>           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0%
>>>> 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>>           File Formats: PARQUET/SNAPPY:30
>>>>           BytesRead(500.000ms): 1.56 MB, 1.56 MB, 1.56 MB
>>>>            - FooterProcessingTime: (Avg: 12.351ms ; Min: 7.816ms ; Max:
>>>> 28.687ms ; Number of samples: 5)
>>>>            - AverageHdfsReadThreadConcurrency: 0.00
>>>>            - AverageScannerThreadConcurrency: 0.00
>>>>            - BytesRead: 1.56 MB (1630546)
>>>>            - BytesReadDataNodeCache: 0
>>>>            - BytesReadLocal: 1.56 MB (1630546)
>>>>            - BytesReadRemoteUnexpected: 0
>>>>            - BytesReadShortCircuit: 1.56 MB (1630546)
>>>>            - CachedFileHandlesHitCount: 0 (0)
>>>>            - CachedFileHandlesMissCount: 35 (35)
>>>>            - DecompressionTime: 1.267ms
>>>>            - MaxCompressedTextFileLength: 0
>>>>            - NumColumns: 6 (6)
>>>>            - NumDictFilteredRowGroups: 0 (0)
>>>>            - NumDisksAccessed: 4 (4)
>>>>            - NumRowGroups: 5 (5)
>>>>            - NumScannerThreadsStarted: 5 (5)
>>>>            - NumScannersWithNoReads: 0 (0)
>>>>            - NumStatsFilteredRowGroups: 0 (0)
>>>>            - PeakMemoryUsage: 5.59 MB (5859290)
>>>>            - PerReadThreadRawHdfsThroughput: 920.32 MB/sec
>>>>            - RemoteScanRanges: 0 (0)
>>>>            - RowBatchQueueGetWaitTime: 15.335ms
>>>>            - RowBatchQueuePutWaitTime: 0.000ns
>>>>            - RowsRead: 64.69K (64692)
>>>>            - RowsReturned: 30.34K (30340)
>>>>            - RowsReturnedRate: 1.79 M/sec
>>>>            - ScanRangesComplete: 5 (5)
>>>>            - ScannerThreadsInvoluntaryContextSwitches: 1 (1)
>>>>            - ScannerThreadsTotalWallClockTime: 132.548ms
>>>>              - MaterializeTupleTime(*): 10.585ms
>>>>              - ScannerThreadsSysTime: 999.000us
>>>>              - ScannerThreadsUserTime: 18.995ms
>>>>            - ScannerThreadsVoluntaryContextSwitches: 79 (79)
>>>>            - TotalRawHdfsReadTime(*): 1.689ms
>>>>            - TotalReadThroughput: 1.04 MB/sec
>>>>       Instance 744de1b6228736fa:b54bfaa700000002
>>>> (host=slpr-dhc003.lpdomain.com:22000):(Total: 1s452ms, non-child: 1s255ms,
%
>>>> non-child: 86.41%)
>>>>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> 0:1/875.15 KB 10:1/927.27 KB 6:1/1.07 MB 9:1/215.54 KB 8:3/1.84 MB
>>>>         MemoryUsage(500.000ms): 5.58 MB, 5.58 MB, 3.94 MB
>>>>         ThreadUsage(500.000ms): 1, 1, 1
>>>>          - AverageThreadTokens: 1.00
>>>>          - BloomFilterBytes: 0
>>>>          - PeakMemoryUsage: 6.17 MB (6473729)
>>>>          - PeakReservation: 0
>>>>          - PeakUsedReservation: 0
>>>>          - PerHostPeakMemUsage: 107.32 MB (112537640)
>>>>          - RowsProduced: 29.97K (29969)
>>>>          - TotalNetworkReceiveTime: 0.000ns
>>>>          - TotalNetworkSendTime: 1s260ms
>>>>          - TotalStorageWaitTime: 130.815ms
>>>>          - TotalThreadsInvoluntaryContextSwitches: 15 (15)
>>>>          - TotalThreadsTotalWallClockTime: 1s586ms
>>>>            - TotalThreadsSysTime: 11.995ms
>>>>            - TotalThreadsUserTime: 146.975ms
>>>>          - TotalThreadsVoluntaryContextSwitches: 197 (197)
>>>>         Fragment Instance Lifecycle Timings:
>>>>            - ExecTime: 1s354ms
>>>>              - ExecTreeExecTime: 67.791ms
>>>>            - OpenTime: 52.012ms
>>>>              - ExecTreeOpenTime: 101.232us
>>>>            - PrepareTime: 45.946ms
>>>>              - ExecTreePrepareTime: 309.644us
>>>>         DataStreamSender (dst_id=11):(Total: 38.995ms, non-child:
>>>> 38.995ms, % non-child: 100.00%)
>>>>            - BytesSent: 35.95 MB (37697260)
>>>>            - NetworkThroughput(*): 2.14 GB/sec
>>>>            - OverallThroughput: 921.92 MB/sec
>>>>            - PeakMemoryUsage: 59.38 KB (60800)
>>>>            - RowsReturned: 29.97K (29969)
>>>>            - SerializeBatchTime: 18.055ms
>>>>            - TransmitDataRPCTime: 16.428ms
>>>>            - UncompressedRowBatchSize: 70.83 MB (74269800)
>>>>         CodeGen:(Total: 90.385ms, non-child: 90.385ms, % non-child:
>>>> 100.00%)
>>>>            - CodegenTime: 2.224ms
>>>>            - CompileTime: 15.517ms
>>>>            - LoadTime: 0.000ns
>>>>            - ModuleBitcodeSize: 1.96 MB (2050180)
>>>>            - NumFunctions: 16 (16)
>>>>            - NumInstructions: 250 (250)
>>>>            - OptimizationTime: 35.099ms
>>>>            - PeakMemoryUsage: 125.00 KB (128000)
>>>>            - PrepareTime: 38.847ms
>>>>         SUBPLAN_NODE (id=6):(Total: 68.126ms, non-child: 12.625ms, %
>>>> non-child: 18.53%)
>>>>            - PeakMemoryUsage: 636.17 KB (651439)
>>>>            - RowsReturned: 29.70K (29696)
>>>>            - RowsReturnedRate: 435.89 K/sec
>>>>           NESTED_LOOP_JOIN_NODE (id=9):(Total: 35.347ms, non-child:
>>>> 27.325ms, % non-child: 77.31%)
>>>>              - BuildRows: 0 (0)
>>>>              - BuildTime: 0.000ns
>>>>              - PeakMemoryUsage: 24.00 KB (24576)
>>>>              - ProbeRows: 29.97K (29969)
>>>>              - ProbeTime: 0.000ns
>>>>              - RowsReturned: 15.26M (15256601)
>>>>              - RowsReturnedRate: 431.62 M/sec
>>>>             Nested Loop Join Builder:
>>>>                - PeakMemoryUsage: 8.00 KB (8192)
>>>>             SINGULAR_ROW_SRC_NODE (id=7):
>>>>                - PeakMemoryUsage: 0
>>>>                - RowsReturned: 0 (0)
>>>>                - RowsReturnedRate: 0
>>>>           UNNEST_NODE (id=8):(Total: 8.021ms, non-child: 8.021ms, %
>>>> non-child: 100.00%)
>>>>              - AvgCollectionSize: 1.00
>>>>              - MaxCollectionSize: 1 (1)
>>>>              - MinCollectionSize: 1 (1)
>>>>              - NumCollections: 29.97K (29969)
>>>>              - PeakMemoryUsage: 0
>>>>              - RowsReturned: 1 (1)
>>>>              - RowsReturnedRate: 124.00 /sec
>>>>         HDFS_SCAN_NODE (id=5):(Total: 20.153ms, non-child: 20.153ms, %
>>>> non-child: 100.00%)
>>>>           Hdfs split stats (<volume id>:<# splits>/<split
lengths>):
>>>> 0:1/875.15 KB 10:1/927.27 KB 6:1/1.07 MB 9:1/215.54 KB 8:3/1.84 MB
>>>>           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 7 out of
>>>> 7
>>>>           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0%
>>>> 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>>           File Formats: PARQUET/SNAPPY:42
>>>>           BytesRead(500.000ms): 1.75 MB, 1.75 MB, 1.75 MB
>>>>            - FooterProcessingTime: (Avg: 8.683ms ; Min: 3.937ms ; Max:
>>>> 10.464ms ; Number of samples: 7)
>>>>            - AverageHdfsReadThreadConcurrency: 0.00
>>>>            - AverageScannerThreadConcurrency: 0.00
>>>>            - BytesRead: 1.75 MB (1833560)
>>>>            - BytesReadDataNodeCache: 0
>>>>            - BytesReadLocal: 1.75 MB (1833560)
>>>>            - BytesReadRemoteUnexpected: 0
>>>>            - BytesReadShortCircuit: 1.75 MB (1833560)
>>>>            - CachedFileHandlesHitCount: 0 (0)
>>>>            - CachedFileHandlesMissCount: 49 (49)
>>>>            - DecompressionTime: 1.758ms
>>>>            - MaxCompressedTextFileLength: 0
>>>>            - NumColumns: 6 (6)
>>>>            - NumDictFilteredRowGroups: 0 (0)
>>>>            - NumDisksAccessed: 5 (5)
>>>>            - NumRowGroups: 7 (7)
>>>>            - NumScannerThreadsStarted: 7 (7)
>>>>            - NumScannersWithNoReads: 0 (0)
>>>>            - NumStatsFilteredRowGroups: 0 (0)
>>>>            - PeakMemoryUsage: 5.82 MB (6098991)
>>>>            - PerReadThreadRawHdfsThroughput: 621.09 MB/sec
>>>>            - RemoteScanRanges: 0 (0)
>>>>            - RowBatchQueueGetWaitTime: 17.938ms
>>>>            - RowBatchQueuePutWaitTime: 0.000ns
>>>>            - RowsRead: 65.17K (65166)
>>>>            - RowsReturned: 29.97K (29969)
>>>>            - RowsReturnedRate: 1.49 M/sec
>>>>            - ScanRangesComplete: 7 (7)
>>>>            - ScannerThreadsInvoluntaryContextSwitches: 1 (1)
>>>>            - ScannerThreadsTotalWallClockTime: 179.698ms
>>>>              - MaterializeTupleTime(*): 13.922ms
>>>>              - ScannerThreadsSysTime: 2.997ms
>>>>              - ScannerThreadsUserTime: 29.993ms
>>>>            - ScannerThreadsVoluntaryContextSwitches: 130 (130)
>>>>            - TotalRawHdfsReadTime(*): 2.815ms
>>>>            - TotalReadThroughput: 1.17 MB/sec
>>>>       Instance 744de1b6228736fa:b54bfaa700000008
>>>> (host=slpr-dhc009.lpdomain.com:22000):(Total: 1s450ms, non-child: 1s305ms,
%
>>>> non-child: 90.01%)
>>>>         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> 7:1/1.08 MB 6:2/2.52 MB 3:2/2.19 MB 8:1/810.47 KB
>>>>         MemoryUsage(500.000ms): 5.97 MB, 5.88 MB, 2.24 MB
>>>>         ThreadUsage(500.000ms): 1, 1, 1
>>>>          - AverageThreadTokens: 1.00
>>>>          - BloomFilterBytes: 0
>>>>          - PeakMemoryUsage: 6.69 MB (7017157)
>>>>          - PeakReservation: 0
>>>>          - PeakUsedReservation: 0
>>>>          - PerHostPeakMemUsage: 106.39 MB (111556990)
>>>>          - RowsProduced: 42.68K (42681)
>>>>          - TotalNetworkReceiveTime: 0.000ns
>>>>          - TotalNetworkSendTime: 1s315ms
>>>>          - TotalStorageWaitTime: 201.411ms
>>>>          - TotalThreadsInvoluntaryContextSwitches: 43 (43)
>>>>          - TotalThreadsTotalWallClockTime: 1s658ms
>>>>            - TotalThreadsSysTime: 6.997ms
>>>>            - TotalThreadsUserTime: 108.980ms
>>>>          - TotalThreadsVoluntaryContextSwitches: 179 (179)
>>>>         Fragment Instance Lifecycle Timings:
>>>>            - ExecTime: 1s411ms
>>>>              - ExecTreeExecTime: 68.017ms
>>>>            - OpenTime: 19.935ms
>>>>              - ExecTreeOpenTime: 48.883us
>>>>            - PrepareTime: 19.160ms
>>>>              - ExecTreePrepareTime: 134.576us
>>>>         DataStreamSender (dst_id=11):(Total: 40.671ms, non-child:
>>>> 40.671ms, % non-child: 100.00%)
>>>>            - BytesSent: 51.36 MB (53856680)
>>>>            - NetworkThroughput(*): 2.44 GB/sec
>>>>            - OverallThroughput: 1.23 GB/sec
>>>>            - PeakMemoryUsage: 59.38 KB (60800)
>>>>            - RowsReturned: 42.68K (42681)
>>>>            - SerializeBatchTime: 18.299ms
>>>>            - TransmitDataRPCTime: 20.591ms
>>>>            - UncompressedRowBatchSize: 100.88 MB (105782860)
>>>>         CodeGen:(Total: 36.171ms, non-child: 36.171ms, % non-child:
>>>> 100.00%)
>>>>            - CodegenTime: 828.865us
>>>>            - CompileTime: 6.170ms
>>>>            - LoadTime: 0.000ns
>>>>            - ModuleBitcodeSize: 1.96 MB (2050180)
>>>>            - NumFunctions: 16 (16)
>>>>            - NumInstructions: 250 (250)
>>>>            - OptimizationTime: 12.988ms
>>>>            - PeakMemoryUsage: 125.00 KB (128000)
>>>>            - PrepareTime: 16.608ms
>>>>         SUBPLAN_NODE (id=6):(Total: 68.046ms, non-child: 12.434ms, %
>>>> non-child: 18.27%)
>>>>            - PeakMemoryUsage: 689.95 KB (706507)
>>>>            - RowsReturned: 41.98K (41984)
>>>>            - RowsReturnedRate: 616.99 K/sec
>>>>           NESTED_LOOP_JOIN_NODE (id=9):(Total: 39.837ms, non-child:
>>>> 29.422ms, % non-child: 73.86%)
>>>>              - BuildRows: 0 (0)
>>>>              - BuildTime: 0.000ns
>>>>              - PeakMemoryUsage: 24.00 KB (24576)
>>>>              - ProbeRows: 42.68K (42681)
>>>>              - ProbeTime: 0.000ns
>>>>              - RowsReturned: 21.76M (21760053)
>>>>              - RowsReturnedRate: 546.22 M/sec
>>>>             Nested Loop Join Builder:
>>>>                - PeakMemoryUsage: 8.00 KB (8192)
>>>>             SINGULAR_ROW_SRC_NODE (id=7):
>>>>                - PeakMemoryUsage: 0
>>>>                - RowsReturned: 0 (0)
>>>>                - RowsReturnedRate: 0
>>>>           UNNEST_NODE (id=8):(Total: 10.415ms, non-child: 10.415ms, %
>>>> non-child: 100.00%)
>>>>              - AvgCollectionSize: 1.00
>>>>              - MaxCollectionSize: 1 (1)
>>>>              - MinCollectionSize: 1 (1)
>>>>              - NumCollections: 42.68K (42681)
>>>>              - PeakMemoryUsage: 0
>>>>              - RowsReturned: 1 (1)
>>>>              - RowsReturnedRate: 96.00 /sec
>>>>         HDFS_SCAN_NODE (id=5):(Total: 15.774ms, non-child: 15.774ms, %
>>>> non-child: 100.00%)
>>>>           Hdfs split stats (<volume id>:<# splits>/<split
lengths>):
>>>> 7:1/1.08 MB 6:2/2.52 MB 3:2/2.19 MB 8:1/810.47 KB
>>>>           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 6 out of
>>>> 6
>>>>           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0%
>>>> 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>> 17:0%
>>>>           File Formats: PARQUET/SNAPPY:36
>>>>           BytesRead(500.000ms): 2.08 MB, 2.08 MB, 2.08 MB
>>>>            - FooterProcessingTime: (Avg: 22.052ms ; Min: 7.783ms ; Max:
>>>> 42.732ms ; Number of samples: 6)
>>>>            - AverageHdfsReadThreadConcurrency: 0.00
>>>>            - AverageScannerThreadConcurrency: 0.00
>>>>            - BytesRead: 2.08 MB (2186159)
>>>>            - BytesReadDataNodeCache: 0
>>>>            - BytesReadLocal: 2.08 MB (2186159)
>>>>            - BytesReadRemoteUnexpected: 0
>>>>            - BytesReadShortCircuit: 2.08 MB (2186159)
>>>>            - CachedFileHandlesHitCount: 0 (0)
>>>>            - CachedFileHandlesMissCount: 42 (42)
>>>>            - DecompressionTime: 1.480ms
>>>>            - MaxCompressedTextFileLength: 0
>>>>            - NumColumns: 6 (6)
>>>>            - NumDictFilteredRowGroups: 0 (0)
>>>>            - NumDisksAccessed: 4 (4)
>>>>            - NumRowGroups: 6 (6)
>>>>            - NumScannerThreadsStarted: 6 (6)
>>>>            - NumScannersWithNoReads: 0 (0)
>>>>            - NumStatsFilteredRowGroups: 0 (0)
>>>>            - PeakMemoryUsage: 6.42 MB (6730467)
>>>>            - PerReadThreadRawHdfsThroughput: 77.02 MB/sec
>>>>            - RemoteScanRanges: 0 (0)
>>>>            - RowBatchQueueGetWaitTime: 13.979ms
>>>>            - RowBatchQueuePutWaitTime: 0.000ns
>>>>            - RowsRead: 89.91K (89910)
>>>>            - RowsReturned: 42.68K (42681)
>>>>            - RowsReturnedRate: 2.71 M/sec
>>>>            - ScanRangesComplete: 6 (6)
>>>>            - ScannerThreadsInvoluntaryContextSwitches: 3 (3)
>>>>            - ScannerThreadsTotalWallClockTime: 227.825ms
>>>>

Mime
View raw message