impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Jacobs <jacobs...@gmail.com>
Subject Re: impala submission control
Date Thu, 08 Feb 2018 18:12:11 GMT
As I said, I don't see anything wrong with the configuration so there could
be a bug in Impala 2.11, likely due to IMPALA-5920. Can you file a JIRA?

On Thu, Feb 8, 2018 at 9:42 AM, Fawze Abujaber <fawzeaj@gmail.com> wrote:

> This is not fitting my situation,
>
> I removed even the placement role of the secondary group, and other users
> can still submit to the pool that i restrict it to my user.
>
> Just thinking in loud and think the problem might be in using the
> placement rule:
>
> Use the pool Specified at run time., only if the pool exists.
> Edit
>
>
> as it it the first rule, it always successes.
>
>
> On Thu, Feb 8, 2018 at 7:12 PM, Matthew Jacobs <jacobs.mh@gmail.com>
> wrote:
>
>> The aclSubmitApps can specify users or groups, so if the user name is
>> also a group name, that can lead to confusion, e.g.
>> https://issues.apache.org/jira/browse/IMPALA-2578
>> I'm not saying that is necessarily the problem, but something to
>> check. If we still can't find anything configured incorrectly I'd
>> recommend that you file a JIRA.
>>
>> On Thu, Feb 8, 2018 at 9:05 AM, Fawze Abujaber <fawzeaj@gmail.com> wrote:
>> > Hi Matt,
>> >
>> > If this is the situation, how we can explain that when i submit the
>> query it
>> > runs on my secondary group pool and when nira do the same it runs under
>> her
>> > secondary group pool.
>> >
>> > More than this, the issue not related to LDAP group as i'm using the
>> > submission restriction to a user and not a group.
>> >
>> > On Thu, Feb 8, 2018 at 7:00 PM, Matthew Jacobs <jacobs.mh@gmail.com>
>> wrote:
>> >>
>> >> As far as I can tell from this information, your configuration looks
>> >> fine. The only other potential issue I can think of is that the
>> >> secondary groups placement rule isn't picking up your LDAP groups
>> >> properly. Does the ACL behavior work if you remove the secondary
>> >> groups rule?
>> >>
>> >> On Wed, Feb 7, 2018 at 10:09 PM, Fawze Abujaber <fawzeaj@gmail.com>
>> wrote:
>> >> > Hi Matt,
>> >> >
>> >> > v2.11.0-cdh5.14.0
>> >> >
>> >> > Query Runtime Profile:
>> >> > Query (id=c843dae48ce9b289:6ad3ffc000000000):
>> >> >   Summary:
>> >> >     Session ID: ca420ac477772b56:51ae460ce9e44999
>> >> >     Session Type: BEESWAX
>> >> >     Start Time: 2018-02-08 01:02:01.023558000
>> >> >     End Time: 2018-02-08 01:02:01.501506000
>> >> >     Query Type: QUERY
>> >> >     Query State: FINISHED
>> >> >     Query Status: OK
>> >> >     Impala Version: impalad version 2.11.0-cdh5.14.0 RELEASE (build
>> >> > d68206561bce6b26762d62c01a78e6cd27aa7690)
>> >> >     User: fawzea
>> >> >     Connected User: fawzea
>> >> >     Delegated User:
>> >> >     Network Address: ::ffff:10.16.144.131:50078
>> >> >     Default Db: default
>> >> >     Sql Statement: select count(*) from dv_entersiteevent_prq_remote
>> >> > where
>> >> > year=2017 and month=10
>> >> >     Coordinator: aopr-dhc001.lpdomain.com:22000
>> >> >     Query Options (set by configuration): REQUEST_POOL=impreports
>> >> >     Query Options (set by configuration and planner):
>> >> > REQUEST_POOL=impreports,MT_DOP=0
>> >> >     Plan:
>> >> > ----------------
>> >> > Max Per-Host Resource Reservation: Memory=0B
>> >> > Per-Host Resource Estimates: Memory=21.00MB
>> >> > WARNING: The following tables are missing relevant table and/or
>> column
>> >> > statistics.
>> >> > default.dv_entersiteevent_prq_remote
>> >> >
>> >> > F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
>> >> > |  Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
>> >> > PLAN-ROOT SINK
>> >> > |  mem-estimate=0B mem-reservation=0B
>> >> > |
>> >> > 03:AGGREGATE [FINALIZE]
>> >> > |  output: count:merge(*)
>> >> > |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
>> >> > |  tuple-ids=1 row-size=8B cardinality=1
>> >> > |
>> >> > 02:EXCHANGE [UNPARTITIONED]
>> >> > |  mem-estimate=0B mem-reservation=0B
>> >> > |  tuple-ids=1 row-size=8B cardinality=1
>> >> > |
>> >> > F00:PLAN FRAGMENT [RANDOM] hosts=5 instances=5
>> >> > Per-Host Resources: mem-estimate=11.00MB mem-reservation=0B
>> >> > 01:AGGREGATE
>> >> > |  output:
>> >> > sum_init_zero(default.dv_entersiteevent_prq_remote.parquet-stats:
>> >> > num_rows)
>> >> > |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
>> >> > |  tuple-ids=1 row-size=8B cardinality=1
>> >> > |
>> >> > 00:SCAN HDFS [default.dv_entersiteevent_prq_remote, RANDOM]
>> >> >    partitions=31/441 files=2292 size=1.39GB
>> >> >    stats-rows=unavailable extrapolated-rows=disabled
>> >> >    table stats: rows=unavailable size=unavailable
>> >> >    column stats: all
>> >> >    mem-estimate=1.00MB mem-reservation=0B
>> >> >    tuple-ids=0 row-size=8B cardinality=unavailable
>> >> > ----------------
>> >> >     Estimated Per-Host Mem: 22020096
>> >> >     Tables Missing Stats: default.dv_entersiteevent_prq_remote
>> >> >     Per Host Min Reservation: aopr-dhc001.lpdomain.com:22000(0)
>> >> > aopr-dhc002.lpdomain.com:22000(0) aopr-dhc003.lpdomain.com:22000(0)
>> >> > aopr-dhc004.lpdomain.com:22000(0) aopr-dhc005.lpdomain.com:22000(0)
>> >> >     Request Pool: root.impreports
>> >> >     Admission result: Admitted immediately
>> >> >     ExecSummary:
>> >> > Operator       #Hosts   Avg Time   Max Time  #Rows  Est. #Rows  Peak
>> Mem
>> >> > Est. Peak Mem  Detail
>> >> >
>> >> > ------------------------------------------------------------
>> ------------------------------------------------------------------
>> >> > 03:AGGREGATE        1  222.958us  222.958us      1           1
>> 36.00 KB
>> >> > 10.00 MB  FINALIZE
>> >> > 02:EXCHANGE         1  357.508ms  357.508ms      5           1
>>    0
>> >> > 0  UNPARTITIONED
>> >> > 01:AGGREGATE        5    1.027ms    1.541ms      5           1
>> 20.00 KB
>> >> > 10.00 MB
>> >> > 00:SCAN HDFS        5  282.797ms  333.715ms  2.29K          -1
>>  2.62 MB
>> >> > 1.00 MB  default.dv_entersiteevent_prq_remote
>> >> >     Errors:
>> >> >     Planner Timeline: 13.508ms
>> >> >        - Analysis finished: 3.303ms (3.303ms)
>> >> >        - Value transfer graph computed: 3.465ms (162.667us)
>> >> >        - Single node plan created: 10.352ms (6.886ms)
>> >> >        - Runtime filters computed: 10.677ms (325.364us)
>> >> >        - Distributed plan created: 10.733ms (55.854us)
>> >> >        - Lineage info computed: 10.898ms (164.937us)
>> >> >        - Planning finished: 13.508ms (2.610ms)
>> >> >     Query Timeline: 481.950ms
>> >> >        - Query submitted: 79.971us (79.971us)
>> >> >        - Planning finished: 33.947ms (33.867ms)
>> >> >        - Submit for admission: 40.067ms (6.120ms)
>> >> >        - Completed admission: 40.197ms (130.146us)
>> >> >        - Ready to start on 5 backends: 40.905ms (707.852us)
>> >> >        - All 5 execution backends (6 fragment instances) started:
>> >> > 46.417ms
>> >> > (5.512ms)
>> >> >        - Rows available: 424.548ms (378.131ms)
>> >> >        - First row fetched: 473.488ms (48.939ms)
>> >> >        - Unregister query: 477.951ms (4.462ms)
>> >> >      - ComputeScanRangeAssignmentTimer: 3.980ms
>> >> >   ImpalaServer:
>> >> >      - ClientFetchWaitTimer: 51.659ms
>> >> >      - RowMaterializationTimer: 1.726ms
>> >> >   Execution Profile c843dae48ce9b289:6ad3ffc000000000:(Total:
>> 385.029ms,
>> >> > non-child: 0.000ns, % non-child: 0.00%)
>> >> >     Number of filters: 0
>> >> >     Filter routing table:
>> >> >  ID  Src. Node  Tgt. Node(s)  Target type  Partition filter  Pending
>> >> > (Expected)  First arrived  Completed   Enabled
>> >> >
>> >> > ------------------------------------------------------------
>> -------------------------------------------------------
>> >> >
>> >> >     Backend startup latencies: Count: 5, min / max: 3ms / 4ms, 25th
>> >> > %-ile:
>> >> > 3ms, 50th %-ile: 4ms, 75th %-ile: 4ms, 90th %-ile: 4ms, 95th %-ile:
>> 4ms,
>> >> > 99.9th %-ile: 4ms
>> >> >     Per Node Peak Memory Usage: aopr-dhc001.lpdomain.com:22000(2.68
>> MB)
>> >> > aopr-dhc002.lpdomain.com:22000(2.54 MB)
>> >> > aopr-dhc003.lpdomain.com:22000(2.54
>> >> > MB) aopr-dhc004.lpdomain.com:22000(2.41 MB)
>> >> > aopr-dhc005.lpdomain.com:22000(2.56 MB)
>> >> >      - FiltersReceived: 0 (0)
>> >> >      - FinalizationTimer: 0.000ns
>> >> >      - NumBackends: 5 (5)
>> >> >      - NumFragmentInstances: 6 (6)
>> >> >      - NumFragments: 2 (2)
>> >> >     Averaged Fragment F01:(Total: 427.530ms, non-child: 49.428ms, %
>> >> > non-child: 11.56%)
>> >> >       split sizes:  min: 0, max: 0, avg: 0, stddev: 0
>> >> >       completion times: min:428.693ms  max:428.693ms  mean: 428.693ms
>> >> > stddev:0.000ns
>> >> >       execution rates: min:0.00 /sec  max:0.00 /sec  mean:0.00 /sec
>> >> > stddev:0.00 /sec
>> >> >       num instances: 1
>> >> >        - AverageThreadTokens: 0.00
>> >> >        - BloomFilterBytes: 0
>> >> >        - PeakMemoryUsage: 44.02 KB (45072)
>> >> >        - PeakReservation: 0
>> >> >        - PeakUsedReservation: 0
>> >> >        - PerHostPeakMemUsage: 2.68 MB (2806113)
>> >> >        - RowsProduced: 1 (1)
>> >> >        - TotalNetworkReceiveTime: 357.481ms
>> >> >        - TotalNetworkSendTime: 0.000ns
>> >> >        - TotalStorageWaitTime: 0.000ns
>> >> >        - TotalThreadsInvoluntaryContextSwitches: 0 (0)
>> >> >        - TotalThreadsTotalWallClockTime: 406.708ms
>> >> >          - TotalThreadsSysTime: 0.000ns
>> >> >          - TotalThreadsUserTime: 0.000ns
>> >> >        - TotalThreadsVoluntaryContextSwitches: 7 (7)
>> >> >       Fragment Instance Lifecycle Timings:
>> >> >          - ExecTime: 49.055ms
>> >> >            - ExecTreeExecTime: 4.251us
>> >> >          - OpenTime: 357.663ms
>> >> >            - ExecTreeOpenTime: 357.649ms
>> >> >          - PrepareTime: 20.764ms
>> >> >            - ExecTreePrepareTime: 81.759us
>> >> >       PLAN_ROOT_SINK:
>> >> >          - PeakMemoryUsage: 0
>> >> >       CodeGen:(Total: 20.369ms, non-child: 20.369ms, % non-child:
>> >> > 100.00%)
>> >> >          - CodegenTime: 0.000ns
>> >> >          - CompileTime: 0.000ns
>> >> >          - LoadTime: 0.000ns
>> >> >          - ModuleBitcodeSize: 1.86 MB (1950880)
>> >> >          - NumFunctions: 0 (0)
>> >> >          - NumInstructions: 0 (0)
>> >> >          - OptimizationTime: 0.000ns
>> >> >          - PeakMemoryUsage: 0
>> >> >          - PrepareTime: 19.685ms
>> >> >       AGGREGATION_NODE (id=3):(Total: 357.731ms, non-child:
>> 222.958us, %
>> >> > non-child: 0.06%)
>> >> >          - BuildTime: 7.368us
>> >> >          - GetResultsTime: 0.000ns
>> >> >          - HTResizeTime: 0.000ns
>> >> >          - HashBuckets: 0 (0)
>> >> >          - LargestPartitionPercent: 0 (0)
>> >> >          - MaxPartitionLevel: 0 (0)
>> >> >          - NumRepartitions: 0 (0)
>> >> >          - PartitionsCreated: 0 (0)
>> >> >          - PeakMemoryUsage: 36.00 KB (36864)
>> >> >          - RowsRepartitioned: 0 (0)
>> >> >          - RowsReturned: 1 (1)
>> >> >          - RowsReturnedRate: 2.00 /sec
>> >> >          - SpilledPartitions: 0 (0)
>> >> >       EXCHANGE_NODE (id=2):(Total: 357.508ms, non-child: 357.508ms, %
>> >> > non-child: 100.00%)
>> >> >          - ConvertRowBatchTime: 8.881us
>> >> >          - PeakMemoryUsage: 0
>> >> >          - RowsReturned: 5 (5)
>> >> >          - RowsReturnedRate: 13.00 /sec
>> >> >         DataStreamReceiver:
>> >> >            - BytesReceived: 80.00 B (80)
>> >> >            - DeserializeRowBatchTimer: 24.165us
>> >> >            - FirstBatchArrivalWaitTime: 277.194ms
>> >> >            - PeakMemoryUsage: 4.02 KB (4112)
>> >> >            - SendersBlockedTimer: 0.000ns
>> >> >            - SendersBlockedTotalTimer(*): 0.000ns
>> >> >     Coordinator Fragment F01:
>> >> >       Instance c843dae48ce9b289:6ad3ffc000000000
>> >> > (host=aopr-dhc001.lpdomain.com:22000):(Total: 427.530ms, non-child:
>> >> > 49.428ms, % non-child: 11.56%)
>> >> >          - AverageThreadTokens: 0.00
>> >> >          - BloomFilterBytes: 0
>> >> >          - PeakMemoryUsage: 44.02 KB (45072)
>> >> >          - PeakReservation: 0
>> >> >          - PeakUsedReservation: 0
>> >> >          - PerHostPeakMemUsage: 2.68 MB (2806113)
>> >> >          - RowsProduced: 1 (1)
>> >> >          - TotalNetworkReceiveTime: 357.481ms
>> >> >          - TotalNetworkSendTime: 0.000ns
>> >> >          - TotalStorageWaitTime: 0.000ns
>> >> >          - TotalThreadsInvoluntaryContextSwitches: 0 (0)
>> >> >          - TotalThreadsTotalWallClockTime: 406.708ms
>> >> >            - TotalThreadsSysTime: 0.000ns
>> >> >            - TotalThreadsUserTime: 0.000ns
>> >> >          - TotalThreadsVoluntaryContextSwitches: 7 (7)
>> >> >         Fragment Instance Lifecycle Timings:
>> >> >            - ExecTime: 49.055ms
>> >> >              - ExecTreeExecTime: 4.251us
>> >> >            - OpenTime: 357.663ms
>> >> >              - ExecTreeOpenTime: 357.649ms
>> >> >            - PrepareTime: 20.764ms
>> >> >              - ExecTreePrepareTime: 81.759us
>> >> >         PLAN_ROOT_SINK:
>> >> >            - PeakMemoryUsage: 0
>> >> >         CodeGen:(Total: 20.369ms, non-child: 20.369ms, % non-child:
>> >> > 100.00%)
>> >> >            - CodegenTime: 0.000ns
>> >> >            - CompileTime: 0.000ns
>> >> >            - LoadTime: 0.000ns
>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>> >> >            - NumFunctions: 0 (0)
>> >> >            - NumInstructions: 0 (0)
>> >> >            - OptimizationTime: 0.000ns
>> >> >            - PeakMemoryUsage: 0
>> >> >            - PrepareTime: 19.685ms
>> >> >         AGGREGATION_NODE (id=3):(Total: 357.731ms, non-child:
>> 222.958us,
>> >> > %
>> >> > non-child: 0.06%)
>> >> >            - BuildTime: 7.368us
>> >> >            - GetResultsTime: 0.000ns
>> >> >            - HTResizeTime: 0.000ns
>> >> >            - HashBuckets: 0 (0)
>> >> >            - LargestPartitionPercent: 0 (0)
>> >> >            - MaxPartitionLevel: 0 (0)
>> >> >            - NumRepartitions: 0 (0)
>> >> >            - PartitionsCreated: 0 (0)
>> >> >            - PeakMemoryUsage: 36.00 KB (36864)
>> >> >            - RowsRepartitioned: 0 (0)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - RowsReturnedRate: 2.00 /sec
>> >> >            - SpilledPartitions: 0 (0)
>> >> >         EXCHANGE_NODE (id=2):(Total: 357.508ms, non-child:
>> 357.508ms, %
>> >> > non-child: 100.00%)
>> >> >            - ConvertRowBatchTime: 8.881us
>> >> >            - PeakMemoryUsage: 0
>> >> >            - RowsReturned: 5 (5)
>> >> >            - RowsReturnedRate: 13.00 /sec
>> >> >           DataStreamReceiver:
>> >> >              - BytesReceived: 80.00 B (80)
>> >> >              - DeserializeRowBatchTimer: 24.165us
>> >> >              - FirstBatchArrivalWaitTime: 277.194ms
>> >> >              - PeakMemoryUsage: 4.02 KB (4112)
>> >> >              - SendersBlockedTimer: 0.000ns
>> >> >              - SendersBlockedTotalTimer(*): 0.000ns
>> >> >     Averaged Fragment F00:(Total: 327.360ms, non-child: 1.578ms, %
>> >> > non-child: 0.48%)
>> >> >       split sizes:  min: 283.31 MB, max: 284.33 MB, avg: 284.04 MB,
>> >> > stddev:
>> >> > 382.00 KB
>> >> >       completion times: min:299.457ms  max:378.950ms  mean: 328.881ms
>> >> > stddev:31.507ms
>> >> >       execution rates: min:750.18 MB/sec  max:949.01 MB/sec
>> mean:871.27
>> >> > MB/sec  stddev:79.74 MB/sec
>> >> >       num instances: 5
>> >> >        - AverageThreadTokens: 15.00
>> >> >        - BloomFilterBytes: 0
>> >> >        - PeakMemoryUsage: 2.54 MB (2666852)
>> >> >        - PeakReservation: 0
>> >> >        - PeakUsedReservation: 0
>> >> >        - PerHostPeakMemUsage: 2.55 MB (2669309)
>> >> >        - RowsProduced: 1 (1)
>> >> >        - TotalNetworkReceiveTime: 0.000ns
>> >> >        - TotalNetworkSendTime: 303.541us
>> >> >        - TotalStorageWaitTime: 4s821ms
>> >> >        - TotalThreadsInvoluntaryContextSwitches: 104 (104)
>> >> >        - TotalThreadsTotalWallClockTime: 6s835ms
>> >> >          - TotalThreadsSysTime: 31.784ms
>> >> >          - TotalThreadsUserTime: 682.084ms
>> >> >        - TotalThreadsVoluntaryContextSwitches: 2.12K (2121)
>> >> >       Fragment Instance Lifecycle Timings:
>> >> >          - ExecTime: 500.816us
>> >> >            - ExecTreeExecTime: 8.200us
>> >> >          - OpenTime: 302.848ms
>> >> >            - ExecTreeOpenTime: 281.309ms
>> >> >          - PrepareTime: 23.946ms
>> >> >            - ExecTreePrepareTime: 2.513ms
>> >> >       DataStreamSender (dst_id=2):(Total: 269.902us, non-child:
>> >> > 269.902us, %
>> >> > non-child: 100.00%)
>> >> >          - BytesSent: 16.00 B (16)
>> >> >          - NetworkThroughput(*): 373.13 KB/sec
>> >> >          - OverallThroughput: 63.53 KB/sec
>> >> >          - PeakMemoryUsage: 16.00 KB (16384)
>> >> >          - RowsReturned: 1 (1)
>> >> >          - SerializeBatchTime: 14.525us
>> >> >          - TransmitDataRPCTime: 43.122us
>> >> >          - UncompressedRowBatchSize: 16.00 B (16)
>> >> >       CodeGen:(Total: 41.687ms, non-child: 41.687ms, % non-child:
>> >> > 100.00%)
>> >> >          - CodegenTime: 1.194ms
>> >> >          - CompileTime: 7.360ms
>> >> >          - LoadTime: 0.000ns
>> >> >          - ModuleBitcodeSize: 1.86 MB (1950880)
>> >> >          - NumFunctions: 15 (15)
>> >> >          - NumInstructions: 214 (214)
>> >> >          - OptimizationTime: 13.529ms
>> >> >          - PeakMemoryUsage: 107.00 KB (109568)
>> >> >          - PrepareTime: 20.084ms
>> >> >       AGGREGATION_NODE (id=1):(Total: 283.825ms, non-child: 1.027ms,
>> %
>> >> > non-child: 0.36%)
>> >> >          - BuildTime: 1.259ms
>> >> >          - GetResultsTime: 0.000ns
>> >> >          - HTResizeTime: 0.000ns
>> >> >          - HashBuckets: 0 (0)
>> >> >          - LargestPartitionPercent: 0 (0)
>> >> >          - MaxPartitionLevel: 0 (0)
>> >> >          - NumRepartitions: 0 (0)
>> >> >          - PartitionsCreated: 0 (0)
>> >> >          - PeakMemoryUsage: 20.00 KB (20480)
>> >> >          - RowsRepartitioned: 0 (0)
>> >> >          - RowsReturned: 1 (1)
>> >> >          - RowsReturnedRate: 2.00 /sec
>> >> >          - SpilledPartitions: 0 (0)
>> >> >       HDFS_SCAN_NODE (id=0):(Total: 282.797ms, non-child: 282.797ms,
>> %
>> >> > non-child: 100.00%)
>> >> >          - AverageHdfsReadThreadConcurrency: 0.00
>> >> >          - AverageScannerThreadConcurrency: 14.40
>> >> >          - BytesRead: 44.52 MB (46686028)
>> >> >          - BytesReadDataNodeCache: 0
>> >> >          - BytesReadLocal: 44.52 MB (46686028)
>> >> >          - BytesReadRemoteUnexpected: 0
>> >> >          - BytesReadShortCircuit: 44.52 MB (46686028)
>> >> >          - CachedFileHandlesHitCount: 0 (0)
>> >> >          - CachedFileHandlesMissCount: 458 (458)
>> >> >          - CollectionItemsRead: 0 (0)
>> >> >          - DecompressionTime: 0.000ns
>> >> >          - MaxCompressedTextFileLength: 0
>> >> >          - NumColumns: 0 (0)
>> >> >          - NumDictFilteredRowGroups: 0 (0)
>> >> >          - NumDisksAccessed: 12 (12)
>> >> >          - NumRowGroups: 458 (458)
>> >> >          - NumScannerThreadsStarted: 24 (24)
>> >> >          - NumScannersWithNoReads: 0 (0)
>> >> >          - NumStatsFilteredRowGroups: 0 (0)
>> >> >          - PeakMemoryUsage: 2.50 MB (2625539)
>> >> >          - PerReadThreadRawHdfsThroughput: 1.29 GB/sec
>> >> >          - RemoteScanRanges: 0 (0)
>> >> >          - RowBatchQueueGetWaitTime: 269.787ms
>> >> >          - RowBatchQueuePutWaitTime: 0.000ns
>> >> >          - RowsRead: 0 (0)
>> >> >          - RowsReturned: 458 (458)
>> >> >          - RowsReturnedRate: 1.64 K/sec
>> >> >          - ScanRangesComplete: 458 (458)
>> >> >          - ScannerThreadsInvoluntaryContextSwitches: 99 (99)
>> >> >          - ScannerThreadsTotalWallClockTime: 6s532ms
>> >> >            - MaterializeTupleTime(*): 70.664us
>> >> >            - ScannerThreadsSysTime: 26.184ms
>> >> >            - ScannerThreadsUserTime: 650.089ms
>> >> >          - ScannerThreadsVoluntaryContextSwitches: 1.67K (1674)
>> >> >          - TotalRawHdfsReadTime(*): 34.416ms
>> >> >          - TotalReadThroughput: 36.86 MB/sec
>> >> >     Fragment F00:
>> >> >       Instance c843dae48ce9b289:6ad3ffc000000004
>> >> > (host=aopr-dhc004.lpdomain.com:22000):(Total: 377.384ms, non-child:
>> >> > 1.619ms,
>> >> > % non-child: 0.43%)
>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 0:36/21.95 MB 11:33
>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/19.87 MB 9:27
>> <https://maps.google.com/?q=MB+9:27&entry=gmail&source=g>/15.16 MB 5:44
>> <https://maps.google.com/?q=MB+5:44&entry=gmail&source=g>/28.07 MB 6:56
>> <https://maps.google.com/?q=MB+6:56&entry=gmail&source=g>/33.32 MB
>> >> > 1:38/22.24 MB 10:44
>> <https://maps.google.com/?q=MB+10:44&entry=gmail&source=g>/27.90 MB 3:34
>> <https://maps.google.com/?q=MB+3:34&entry=gmail&source=g>/20.26 MB 2:45
>> <https://maps.google.com/?q=MB+2:45&entry=gmail&source=g>/28.21 MB 7:31
>> <https://maps.google.com/?q=MB+7:31&entry=gmail&source=g>/19.33 MB
>> >> > 4:42/26.73 MB 8:34
>> <https://maps.google.com/?q=MB+8:34&entry=gmail&source=g>/21.25 MB
>> >> >         MemoryUsage(500.000ms): 1.91 MB
>> >> >         ThreadUsage(500.000ms): 25
>> >> >          - AverageThreadTokens: 25.00
>> >> >          - BloomFilterBytes: 0
>> >> >          - PeakMemoryUsage: 2.41 MB (2531689)
>> >> >          - PeakReservation: 0
>> >> >          - PeakUsedReservation: 0
>> >> >          - PerHostPeakMemUsage: 2.41 MB (2531689)
>> >> >          - RowsProduced: 1 (1)
>> >> >          - TotalNetworkReceiveTime: 0.000ns
>> >> >          - TotalNetworkSendTime: 382.629us
>> >> >          - TotalStorageWaitTime: 5s790ms
>> >> >          - TotalThreadsInvoluntaryContextSwitches: 115 (115)
>> >> >          - TotalThreadsTotalWallClockTime: 8s122ms
>> >> >            - TotalThreadsSysTime: 33.984ms
>> >> >            - TotalThreadsUserTime: 702.879ms
>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.04K (2037)
>> >> >         Fragment Instance Lifecycle Timings:
>> >> >            - ExecTime: 604.005us
>> >> >              - ExecTreeExecTime: 8.338us
>> >> >            - OpenTime: 352.927ms
>> >> >              - ExecTreeOpenTime: 331.921ms
>> >> >            - PrepareTime: 23.783ms
>> >> >              - ExecTreePrepareTime: 2.559ms
>> >> >         DataStreamSender (dst_id=2):(Total: 331.818us, non-child:
>> >> > 331.818us,
>> >> > % non-child: 100.00%)
>> >> >            - BytesSent: 16.00 B (16)
>> >> >            - NetworkThroughput(*): 289.00 KB/sec
>> >> >            - OverallThroughput: 47.09 KB/sec
>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - SerializeBatchTime: 14.466us
>> >> >            - TransmitDataRPCTime: 54.066us
>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>> >> >         CodeGen:(Total: 40.950ms, non-child: 40.950ms, % non-child:
>> >> > 100.00%)
>> >> >            - CodegenTime: 1.122ms
>> >> >            - CompileTime: 7.318ms
>> >> >            - LoadTime: 0.000ns
>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>> >> >            - NumFunctions: 15 (15)
>> >> >            - NumInstructions: 214 (214)
>> >> >            - OptimizationTime: 13.106ms
>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>> >> >            - PrepareTime: 19.773ms
>> >> >         AGGREGATION_NODE (id=1):(Total: 334.482ms, non-child:
>> 766.527us,
>> >> > %
>> >> > non-child: 0.23%)
>> >> >           ExecOption: Codegen Enabled
>> >> >            - BuildTime: 1.192ms
>> >> >            - GetResultsTime: 0.000ns
>> >> >            - HTResizeTime: 0.000ns
>> >> >            - HashBuckets: 0 (0)
>> >> >            - LargestPartitionPercent: 0 (0)
>> >> >            - MaxPartitionLevel: 0 (0)
>> >> >            - NumRepartitions: 0 (0)
>> >> >            - PartitionsCreated: 0 (0)
>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>> >> >            - RowsRepartitioned: 0 (0)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - RowsReturnedRate: 2.00 /sec
>> >> >            - SpilledPartitions: 0 (0)
>> >> >         HDFS_SCAN_NODE (id=0):(Total: 333.715ms, non-child:
>> 333.715ms, %
>> >> > non-child: 100.00%)
>> >> >           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 0:36/21.95 MB 11:33
>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/19.87 MB 9:27
>> <https://maps.google.com/?q=MB+9:27&entry=gmail&source=g>/15.16 MB 5:44
>> <https://maps.google.com/?q=MB+5:44&entry=gmail&source=g>/28.07 MB 6:56
>> <https://maps.google.com/?q=MB+6:56&entry=gmail&source=g>/33.32 MB
>> >> > 1:38/22.24 MB 10:44
>> <https://maps.google.com/?q=MB+10:44&entry=gmail&source=g>/27.90 MB 3:34
>> <https://maps.google.com/?q=MB+3:34&entry=gmail&source=g>/20.26 MB 2:45
>> <https://maps.google.com/?q=MB+2:45&entry=gmail&source=g>/28.21 MB 7:31
>> <https://maps.google.com/?q=MB+7:31&entry=gmail&source=g>/19.33 MB
>> >> > 4:42/26.73 MB 8:34
>> <https://maps.google.com/?q=MB+8:34&entry=gmail&source=g>/21.25 MB
>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 464
>> out
>> >> > of
>> >> > 464
>> >> >           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0%
>> >> > 4:0%
>> >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>> >> >           File Formats: PARQUET/Unknown(Skipped):464
>> >> >           BytesRead(500.000ms): 12.27 MB
>> >> >            - FooterProcessingTime: (Avg: 13.555ms ; Min: 5.433ms ;
>> Max:
>> >> > 62.957ms ; Number of samples: 464)
>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>> >> >            - AverageScannerThreadConcurrency: 24.00
>> >> >            - BytesRead: 45.01 MB (47193988)
>> >> >            - BytesReadDataNodeCache: 0
>> >> >            - BytesReadLocal: 45.01 MB (47193988)
>> >> >            - BytesReadRemoteUnexpected: 0
>> >> >            - BytesReadShortCircuit: 45.01 MB (47193988)
>> >> >            - CachedFileHandlesHitCount: 0 (0)
>> >> >            - CachedFileHandlesMissCount: 464 (464)
>> >> >            - CollectionItemsRead: 0 (0)
>> >> >            - DecompressionTime: 0.000ns
>> >> >            - MaxCompressedTextFileLength: 0
>> >> >            - NumColumns: 0 (0)
>> >> >            - NumDictFilteredRowGroups: 0 (0)
>> >> >            - NumDisksAccessed: 12 (12)
>> >> >            - NumRowGroups: 464 (464)
>> >> >            - NumScannerThreadsStarted: 24 (24)
>> >> >            - NumScannersWithNoReads: 0 (0)
>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>> >> >            - PeakMemoryUsage: 2.38 MB (2490376)
>> >> >            - PerReadThreadRawHdfsThroughput: 1016.44 MB/sec
>> >> >            - RemoteScanRanges: 0 (0)
>> >> >            - RowBatchQueueGetWaitTime: 320.860ms
>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>> >> >            - RowsRead: 0 (0)
>> >> >            - RowsReturned: 464 (464)
>> >> >            - RowsReturnedRate: 1.39 K/sec
>> >> >            - ScanRangesComplete: 464 (464)
>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 108 (108)
>> >> >            - ScannerThreadsTotalWallClockTime: 7s769ms
>> >> >              - MaterializeTupleTime(*): 75.514us
>> >> >              - ScannerThreadsSysTime: 29.984ms
>> >> >              - ScannerThreadsUserTime: 669.884ms
>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.60K (1597)
>> >> >            - TotalRawHdfsReadTime(*): 44.279ms
>> >> >            - TotalReadThroughput: 24.53 MB/sec
>> >> >       Instance c843dae48ce9b289:6ad3ffc000000002
>> >> > (host=aopr-dhc003.lpdomain.com:22000):(Total: 351.014ms, non-child:
>> >> > 1.641ms,
>> >> > % non-child: 0.47%)
>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 0:35/20.89 MB 7:44
>> <https://maps.google.com/?q=MB+7:44&entry=gmail&source=g>/26.64 MB 11:27
>> <https://maps.google.com/?q=MB+11:27&entry=gmail&source=g>/17.21 MB 9:32
>> <https://maps.google.com/?q=MB+9:32&entry=gmail&source=g>/18.61 MB 5:25
>> <https://maps.google.com/?q=MB+5:25&entry=gmail&source=g>/14.56 MB
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+2:38&entry=gmail&source=g>
>> >> > 2:38
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+2:38&entry=gmail&source=g>/24.32
>> MB 6:31 <https://maps.google.com/?q=MB+6:31&entry=gmail&source=g>/19.85 MB
>> 8:50 <https://maps.google.com/?q=MB+8:50&entry=gmail&source=g>/30.76 MB
>> 10:41 <https://maps.google.com/?q=MB+10:41&entry=gmail&source=g>/25.92 MB
>> 3:31 <https://maps.google.com/?q=MB+3:31&entry=gmail&source=g>/19.78 MB
>> >> > 4:60/37.59 MB 1:44/27.19 MB
>> >> >         MemoryUsage(500.000ms): 1.16 MB
>> >> >         ThreadUsage(500.000ms): 25
>> >> >          - AverageThreadTokens: 25.00
>> >> >          - BloomFilterBytes: 0
>> >> >          - PeakMemoryUsage: 2.54 MB (2662753)
>> >> >          - PeakReservation: 0
>> >> >          - PeakUsedReservation: 0
>> >> >          - PerHostPeakMemUsage: 2.54 MB (2662753)
>> >> >          - RowsProduced: 1 (1)
>> >> >          - TotalNetworkReceiveTime: 0.000ns
>> >> >          - TotalNetworkSendTime: 342.746us
>> >> >          - TotalStorageWaitTime: 4s796ms
>> >> >          - TotalThreadsInvoluntaryContextSwitches: 93 (93)
>> >> >          - TotalThreadsTotalWallClockTime: 7s286ms
>> >> >            - TotalThreadsSysTime: 27.988ms
>> >> >            - TotalThreadsUserTime: 671.887ms
>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.08K (2079)
>> >> >         Fragment Instance Lifecycle Timings:
>> >> >            - ExecTime: 524.006us
>> >> >              - ExecTreeExecTime: 7.920us
>> >> >            - OpenTime: 327.802ms
>> >> >              - ExecTreeOpenTime: 307.154ms
>> >> >            - PrepareTime: 22.617ms
>> >> >              - ExecTreePrepareTime: 2.650ms
>> >> >         DataStreamSender (dst_id=2):(Total: 348.990us, non-child:
>> >> > 348.990us,
>> >> > % non-child: 100.00%)
>> >> >            - BytesSent: 16.00 B (16)
>> >> >            - NetworkThroughput(*): 354.47 KB/sec
>> >> >            - OverallThroughput: 44.77 KB/sec
>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - SerializeBatchTime: 15.602us
>> >> >            - TransmitDataRPCTime: 44.080us
>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>> >> >         CodeGen:(Total: 39.215ms, non-child: 39.215ms, % non-child:
>> >> > 100.00%)
>> >> >            - CodegenTime: 1.183ms
>> >> >            - CompileTime: 7.168ms
>> >> >            - LoadTime: 0.000ns
>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>> >> >            - NumFunctions: 15 (15)
>> >> >            - NumInstructions: 214 (214)
>> >> >            - OptimizationTime: 12.929ms
>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>> >> >            - PrepareTime: 18.453ms
>> >> >         AGGREGATION_NODE (id=1):(Total: 309.807ms, non-child:
>> 948.677us,
>> >> > %
>> >> > non-child: 0.31%)
>> >> >           ExecOption: Codegen Enabled
>> >> >            - BuildTime: 1.552ms
>> >> >            - GetResultsTime: 0.000ns
>> >> >            - HTResizeTime: 0.000ns
>> >> >            - HashBuckets: 0 (0)
>> >> >            - LargestPartitionPercent: 0 (0)
>> >> >            - MaxPartitionLevel: 0 (0)
>> >> >            - NumRepartitions: 0 (0)
>> >> >            - PartitionsCreated: 0 (0)
>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>> >> >            - RowsRepartitioned: 0 (0)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - RowsReturnedRate: 3.00 /sec
>> >> >            - SpilledPartitions: 0 (0)
>> >> >         HDFS_SCAN_NODE (id=0):(Total: 308.859ms, non-child:
>> 308.859ms, %
>> >> > non-child: 100.00%)
>> >> >           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 0:35/20.89 MB 7:44
>> <https://maps.google.com/?q=MB+7:44&entry=gmail&source=g>/26.64 MB 11:27
>> <https://maps.google.com/?q=MB+11:27&entry=gmail&source=g>/17.21 MB 9:32
>> <https://maps.google.com/?q=MB+9:32&entry=gmail&source=g>/18.61 MB 5:25
>> <https://maps.google.com/?q=MB+5:25&entry=gmail&source=g>/14.56 MB
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+2:38&entry=gmail&source=g>
>> >> > 2:38
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+2:38&entry=gmail&source=g>/24.32
>> MB 6:31 <https://maps.google.com/?q=MB+6:31&entry=gmail&source=g>/19.85 MB
>> 8:50 <https://maps.google.com/?q=MB+8:50&entry=gmail&source=g>/30.76 MB
>> 10:41 <https://maps.google.com/?q=MB+10:41&entry=gmail&source=g>/25.92 MB
>> 3:31 <https://maps.google.com/?q=MB+3:31&entry=gmail&source=g>/19.78 MB
>> >> > 4:60/37.59 MB 1:44/27.19 MB
>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 458
>> out
>> >> > of
>> >> > 458
>> >> >           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0%
>> >> > 4:0%
>> >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>> >> >           File Formats: PARQUET/Unknown(Skipped):458
>> >> >           BytesRead(500.000ms): 36.74 MB
>> >> >            - FooterProcessingTime: (Avg: 11.502ms ; Min: 3.668ms ;
>> Max:
>> >> > 23.564ms ; Number of samples: 458)
>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>> >> >            - AverageScannerThreadConcurrency: 24.00
>> >> >            - BytesRead: 44.51 MB (46668432)
>> >> >            - BytesReadDataNodeCache: 0
>> >> >            - BytesReadLocal: 44.51 MB (46668432)
>> >> >            - BytesReadRemoteUnexpected: 0
>> >> >            - BytesReadShortCircuit: 44.51 MB (46668432)
>> >> >            - CachedFileHandlesHitCount: 0 (0)
>> >> >            - CachedFileHandlesMissCount: 458 (458)
>> >> >            - CollectionItemsRead: 0 (0)
>> >> >            - DecompressionTime: 0.000ns
>> >> >            - MaxCompressedTextFileLength: 0
>> >> >            - NumColumns: 0 (0)
>> >> >            - NumDictFilteredRowGroups: 0 (0)
>> >> >            - NumDisksAccessed: 12 (12)
>> >> >            - NumRowGroups: 458 (458)
>> >> >            - NumScannerThreadsStarted: 24 (24)
>> >> >            - NumScannersWithNoReads: 0 (0)
>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>> >> >            - PeakMemoryUsage: 2.50 MB (2621440)
>> >> >            - PerReadThreadRawHdfsThroughput: 1.26 GB/sec
>> >> >            - RemoteScanRanges: 0 (0)
>> >> >            - RowBatchQueueGetWaitTime: 295.595ms
>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>> >> >            - RowsRead: 0 (0)
>> >> >            - RowsReturned: 458 (458)
>> >> >            - RowsReturnedRate: 1.48 K/sec
>> >> >            - ScanRangesComplete: 458 (458)
>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 91 (91)
>> >> >            - ScannerThreadsTotalWallClockTime: 6s958ms
>> >> >              - MaterializeTupleTime(*): 71.630us
>> >> >              - ScannerThreadsSysTime: 16.989ms
>> >> >              - ScannerThreadsUserTime: 645.891ms
>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.63K (1630)
>> >> >            - TotalRawHdfsReadTime(*): 34.377ms
>> >> >            - TotalReadThroughput: 73.49 MB/sec
>> >> >       Instance c843dae48ce9b289:6ad3ffc000000001
>> >> > (host=aopr-dhc005.lpdomain.com:22000):(Total: 308.195ms, non-child:
>> >> > 1.784ms,
>> >> > % non-child: 0.58%)
>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 2:46/28.97 MB 8:39
>> <https://maps.google.com/?q=MB+8:39&entry=gmail&source=g>/24.34 MB 10:33
>> <https://maps.google.com/?q=MB+10:33&entry=gmail&source=g>/21.03 MB 3:43
>> <https://maps.google.com/?q=MB+3:43&entry=gmail&source=g>/26.50 MB 5:32
>> <https://maps.google.com/?q=MB+5:32&entry=gmail&source=g>/20.27 MB
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>
>> >> > 9:32
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>/19.86
>> MB 6:25 <https://maps.google.com/?q=MB+6:25&entry=gmail&source=g>/16.01 MB
>> 7:41 <https://maps.google.com/?q=MB+7:41&entry=gmail&source=g>/24.77 MB
>> 1:42/25.01 MB 0:55/32.51 MB
>> >> > 4:40/24.29 MB 11:33
>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/20.77 MB
>> >> >          - AverageThreadTokens: 0.00
>> >> >          - BloomFilterBytes: 0
>> >> >          - PeakMemoryUsage: 2.56 MB (2683241)
>> >> >          - PeakReservation: 0
>> >> >          - PeakUsedReservation: 0
>> >> >          - PerHostPeakMemUsage: 2.56 MB (2683241)
>> >> >          - RowsProduced: 1 (1)
>> >> >          - TotalNetworkReceiveTime: 0.000ns
>> >> >          - TotalNetworkSendTime: 292.765us
>> >> >          - TotalStorageWaitTime: 4s482ms
>> >> >          - TotalThreadsInvoluntaryContextSwitches: 113 (113)
>> >> >          - TotalThreadsTotalWallClockTime: 6s310ms
>> >> >            - TotalThreadsSysTime: 35.983ms
>> >> >            - TotalThreadsUserTime: 682.884ms
>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.14K (2143)
>> >> >         Fragment Instance Lifecycle Timings:
>> >> >            - ExecTime: 532.572us
>> >> >              - ExecTreeExecTime: 8.377us
>> >> >            - OpenTime: 280.241ms
>> >> >              - ExecTreeOpenTime: 258.652ms
>> >> >            - PrepareTime: 27.342ms
>> >> >              - ExecTreePrepareTime: 2.863ms
>> >> >         DataStreamSender (dst_id=2):(Total: 309.794us, non-child:
>> >> > 309.794us,
>> >> > % non-child: 100.00%)
>> >> >            - BytesSent: 16.00 B (16)
>> >> >            - NetworkThroughput(*): 356.91 KB/sec
>> >> >            - OverallThroughput: 50.44 KB/sec
>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - SerializeBatchTime: 14.262us
>> >> >            - TransmitDataRPCTime: 43.778us
>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>> >> >         CodeGen:(Total: 44.584ms, non-child: 44.584ms, % non-child:
>> >> > 100.00%)
>> >> >            - CodegenTime: 1.382ms
>> >> >            - CompileTime: 7.346ms
>> >> >            - LoadTime: 0.000ns
>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>> >> >            - NumFunctions: 15 (15)
>> >> >            - NumInstructions: 214 (214)
>> >> >            - OptimizationTime: 13.462ms
>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>> >> >            - PrepareTime: 23.006ms
>> >> >         AGGREGATION_NODE (id=1):(Total: 261.516ms, non-child:
>> 836.656us,
>> >> > %
>> >> > non-child: 0.32%)
>> >> >           ExecOption: Codegen Enabled
>> >> >            - BuildTime: 1.280ms
>> >> >            - GetResultsTime: 0.000ns
>> >> >            - HTResizeTime: 0.000ns
>> >> >            - HashBuckets: 0 (0)
>> >> >            - LargestPartitionPercent: 0 (0)
>> >> >            - MaxPartitionLevel: 0 (0)
>> >> >            - NumRepartitions: 0 (0)
>> >> >            - PartitionsCreated: 0 (0)
>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>> >> >            - RowsRepartitioned: 0 (0)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - RowsReturnedRate: 3.00 /sec
>> >> >            - SpilledPartitions: 0 (0)
>> >> >         HDFS_SCAN_NODE (id=0):(Total: 260.680ms, non-child:
>> 260.680ms, %
>> >> > non-child: 100.00%)
>> >> >           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 2:46/28.97 MB 8:39
>> <https://maps.google.com/?q=MB+8:39&entry=gmail&source=g>/24.34 MB 10:33
>> <https://maps.google.com/?q=MB+10:33&entry=gmail&source=g>/21.03 MB 3:43
>> <https://maps.google.com/?q=MB+3:43&entry=gmail&source=g>/26.50 MB 5:32
>> <https://maps.google.com/?q=MB+5:32&entry=gmail&source=g>/20.27 MB
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>
>> >> > 9:32
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>/19.86
>> MB 6:25 <https://maps.google.com/?q=MB+6:25&entry=gmail&source=g>/16.01 MB
>> 7:41 <https://maps.google.com/?q=MB+7:41&entry=gmail&source=g>/24.77 MB
>> 1:42/25.01 MB 0:55/32.51 MB
>> >> > 4:40/24.29 MB 11:33
>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/20.77 MB
>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 461
>> out
>> >> > of
>> >> > 461
>> >> >           Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0%
>> 4:0%
>> >> > 5:0%
>> >> > 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>> >> >           File Formats: PARQUET/Unknown(Skipped):461
>> >> >            - FooterProcessingTime: (Avg: 10.793ms ; Min: 3.087ms ;
>> Max:
>> >> > 20.253ms ; Number of samples: 461)
>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>> >> >            - AverageScannerThreadConcurrency: 0.00
>> >> >            - BytesRead: 44.73 MB (46906407)
>> >> >            - BytesReadDataNodeCache: 0
>> >> >            - BytesReadLocal: 44.73 MB (46906407)
>> >> >            - BytesReadRemoteUnexpected: 0
>> >> >            - BytesReadShortCircuit: 44.73 MB (46906407)
>> >> >            - CachedFileHandlesHitCount: 0 (0)
>> >> >            - CachedFileHandlesMissCount: 461 (461)
>> >> >            - CollectionItemsRead: 0 (0)
>> >> >            - DecompressionTime: 0.000ns
>> >> >            - MaxCompressedTextFileLength: 0
>> >> >            - NumColumns: 0 (0)
>> >> >            - NumDictFilteredRowGroups: 0 (0)
>> >> >            - NumDisksAccessed: 12 (12)
>> >> >            - NumRowGroups: 461 (461)
>> >> >            - NumScannerThreadsStarted: 24 (24)
>> >> >            - NumScannersWithNoReads: 0 (0)
>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>> >> >            - PeakMemoryUsage: 2.52 MB (2641928)
>> >> >            - PerReadThreadRawHdfsThroughput: 1.31 GB/sec
>> >> >            - RemoteScanRanges: 0 (0)
>> >> >            - RowBatchQueueGetWaitTime: 246.691ms
>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>> >> >            - RowsRead: 0 (0)
>> >> >            - RowsReturned: 461 (461)
>> >> >            - RowsReturnedRate: 1.77 K/sec
>> >> >            - ScanRangesComplete: 461 (461)
>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 109 (109)
>> >> >            - ScannerThreadsTotalWallClockTime: 6s030ms
>> >> >              - MaterializeTupleTime(*): 68.735us
>> >> >              - ScannerThreadsSysTime: 33.983ms
>> >> >              - ScannerThreadsUserTime: 647.889ms
>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.70K (1696)
>> >> >            - TotalRawHdfsReadTime(*): 33.405ms
>> >> >            - TotalReadThroughput: 0.00 /sec
>> >> >       Instance c843dae48ce9b289:6ad3ffc000000003
>> >> > (host=aopr-dhc002.lpdomain.com:22000):(Total: 302.116ms, non-child:
>> >> > 1.604ms,
>> >> > % non-child: 0.53%)
>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 0:27/16.21 MB 11:33
>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/21.23 MB 6:36
>> <https://maps.google.com/?q=MB+6:36&entry=gmail&source=g>/23.15 MB 10:41
>> <https://maps.google.com/?q=MB+10:41&entry=gmail&source=g>/25.09 MB
>> 1:41/24.13 MB
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>
>> >> > 9:32
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>/18.79
>> MB 5:41 <https://maps.google.com/?q=MB+5:41&entry=gmail&source=g>/25.31 MB
>> 2:42 <https://maps.google.com/?q=MB+2:42&entry=gmail&source=g>/26.49 MB
>> 7:36 <https://maps.google.com/?q=MB+7:36&entry=gmail&source=g>/22.87 MB
>> 3:51 <https://maps.google.com/?q=MB+3:51&entry=gmail&source=g>/31.00 MB
>> >> > 4:41/25.99 MB 8:38
>> <https://maps.google.com/?q=MB+8:38&entry=gmail&source=g>/23.79 MB
>> >> >         MemoryUsage(500.000ms): 700.35 KB
>> >> >         ThreadUsage(500.000ms): 25
>> >> >          - AverageThreadTokens: 25.00
>> >> >          - BloomFilterBytes: 0
>> >> >          - PeakMemoryUsage: 2.54 MB (2662753)
>> >> >          - PeakReservation: 0
>> >> >          - PeakUsedReservation: 0
>> >> >          - PerHostPeakMemUsage: 2.54 MB (2662753)
>> >> >          - RowsProduced: 1 (1)
>> >> >          - TotalNetworkReceiveTime: 0.000ns
>> >> >          - TotalNetworkSendTime: 300.675us
>> >> >          - TotalStorageWaitTime: 4s661ms
>> >> >          - TotalThreadsInvoluntaryContextSwitches: 90 (90)
>> >> >          - TotalThreadsTotalWallClockTime: 6s256ms
>> >> >            - TotalThreadsSysTime: 30.983ms
>> >> >            - TotalThreadsUserTime: 709.880ms
>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.17K (2175)
>> >> >         Fragment Instance Lifecycle Timings:
>> >> >            - ExecTime: 492.627us
>> >> >              - ExecTreeExecTime: 8.191us
>> >> >            - OpenTime: 277.922ms
>> >> >              - ExecTreeOpenTime: 255.395ms
>> >> >            - PrepareTime: 23.647ms
>> >> >              - ExecTreePrepareTime: 2.243ms
>> >> >         DataStreamSender (dst_id=2):(Total: 194.517us, non-child:
>> >> > 194.517us,
>> >> > % non-child: 100.00%)
>> >> >            - BytesSent: 16.00 B (16)
>> >> >            - NetworkThroughput(*): 371.86 KB/sec
>> >> >            - OverallThroughput: 80.33 KB/sec
>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - SerializeBatchTime: 14.035us
>> >> >            - TransmitDataRPCTime: 42.019us
>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>> >> >         CodeGen:(Total: 42.676ms, non-child: 42.676ms, % non-child:
>> >> > 100.00%)
>> >> >            - CodegenTime: 1.236ms
>> >> >            - CompileTime: 7.717ms
>> >> >            - LoadTime: 0.000ns
>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>> >> >            - NumFunctions: 15 (15)
>> >> >            - NumInstructions: 214 (214)
>> >> >            - OptimizationTime: 14.130ms
>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>> >> >            - PrepareTime: 20.109ms
>> >> >         AGGREGATION_NODE (id=1):(Total: 257.640ms, non-child:
>> 1.541ms, %
>> >> > non-child: 0.60%)
>> >> >           ExecOption: Codegen Enabled
>> >> >            - BuildTime: 1.252ms
>> >> >            - GetResultsTime: 0.000ns
>> >> >            - HTResizeTime: 0.000ns
>> >> >            - HashBuckets: 0 (0)
>> >> >            - LargestPartitionPercent: 0 (0)
>> >> >            - MaxPartitionLevel: 0 (0)
>> >> >            - NumRepartitions: 0 (0)
>> >> >            - PartitionsCreated: 0 (0)
>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>> >> >            - RowsRepartitioned: 0 (0)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - RowsReturnedRate: 3.00 /sec
>> >> >            - SpilledPartitions: 0 (0)
>> >> >         HDFS_SCAN_NODE (id=0):(Total: 256.099ms, non-child:
>> 256.099ms, %
>> >> > non-child: 100.00%)
>> >> >           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 0:27/16.21 MB 11:33
>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/21.23 MB 6:36
>> <https://maps.google.com/?q=MB+6:36&entry=gmail&source=g>/23.15 MB 10:41
>> <https://maps.google.com/?q=MB+10:41&entry=gmail&source=g>/25.09 MB
>> 1:41/24.13 MB
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>
>> >> > 9:32
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>/18.79
>> MB 5:41 <https://maps.google.com/?q=MB+5:41&entry=gmail&source=g>/25.31 MB
>> 2:42 <https://maps.google.com/?q=MB+2:42&entry=gmail&source=g>/26.49 MB
>> 7:36 <https://maps.google.com/?q=MB+7:36&entry=gmail&source=g>/22.87 MB
>> 3:51 <https://maps.google.com/?q=MB+3:51&entry=gmail&source=g>/31.00 MB
>> >> > 4:41/25.99 MB 8:38
>> <https://maps.google.com/?q=MB+8:38&entry=gmail&source=g>/23.79 MB
>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 459
>> out
>> >> > of
>> >> > 459
>> >> >           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0% 3:0%
>> >> > 4:0%
>> >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>> >> >           File Formats: PARQUET/Unknown(Skipped):459
>> >> >           BytesRead(500.000ms): 43.15 MB
>> >> >            - FooterProcessingTime: (Avg: 11.257ms ; Min: 3.989ms ;
>> Max:
>> >> > 23.004ms ; Number of samples: 459)
>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>> >> >            - AverageScannerThreadConcurrency: 24.00
>> >> >            - BytesRead: 44.52 MB (46678031)
>> >> >            - BytesReadDataNodeCache: 0
>> >> >            - BytesReadLocal: 44.52 MB (46678031)
>> >> >            - BytesReadRemoteUnexpected: 0
>> >> >            - BytesReadShortCircuit: 44.52 MB (46678031)
>> >> >            - CachedFileHandlesHitCount: 0 (0)
>> >> >            - CachedFileHandlesMissCount: 459 (459)
>> >> >            - CollectionItemsRead: 0 (0)
>> >> >            - DecompressionTime: 0.000ns
>> >> >            - MaxCompressedTextFileLength: 0
>> >> >            - NumColumns: 0 (0)
>> >> >            - NumDictFilteredRowGroups: 0 (0)
>> >> >            - NumDisksAccessed: 12 (12)
>> >> >            - NumRowGroups: 459 (459)
>> >> >            - NumScannerThreadsStarted: 24 (24)
>> >> >            - NumScannersWithNoReads: 0 (0)
>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>> >> >            - PeakMemoryUsage: 2.50 MB (2621440)
>> >> >            - PerReadThreadRawHdfsThroughput: 1.33 GB/sec
>> >> >            - RemoteScanRanges: 0 (0)
>> >> >            - RowBatchQueueGetWaitTime: 243.426ms
>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>> >> >            - RowsRead: 0 (0)
>> >> >            - RowsReturned: 459 (459)
>> >> >            - RowsReturnedRate: 1.79 K/sec
>> >> >            - ScanRangesComplete: 459 (459)
>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 85 (85)
>> >> >            - ScannerThreadsTotalWallClockTime: 5s978ms
>> >> >              - MaterializeTupleTime(*): 67.997us
>> >> >              - ScannerThreadsSysTime: 22.984ms
>> >> >              - ScannerThreadsUserTime: 678.884ms
>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.72K (1724)
>> >> >            - TotalRawHdfsReadTime(*): 32.626ms
>> >> >            - TotalReadThroughput: 86.30 MB/sec
>> >> >       Instance c843dae48ce9b289:6ad3ffc000000005
>> >> > (host=aopr-dhc001.lpdomain.com:22000):(Total: 298.093ms, non-child:
>> >> > 1.242ms,
>> >> > % non-child: 0.42%)
>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 9:55/35.38 MB 0:37/24.23 MB 5:37
>> <https://maps.google.com/?q=MB+5:37&entry=gmail&source=g>/23.31 MB 11:22
>> <https://maps.google.com/?q=MB+11:22&entry=gmail&source=g>/13.79 MB 8:30
>> <https://maps.google.com/?q=MB+8:30&entry=gmail&source=g>/18.53 MB
>> >> > 4:33/20.45 MB 1:38/24.06 MB 10:29
>> <https://maps.google.com/?q=MB+10:29&entry=gmail&source=g>/18.40 MB 3:39
>> <https://maps.google.com/?q=MB+3:39&entry=gmail&source=g>/24.35 MB 2:41
>> <https://maps.google.com/?q=MB+2:41&entry=gmail&source=g>/25.45 MB
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+6:42&entry=gmail&source=g>
>> >> > 6:42
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+6:42&entry=gmail&source=g>/27.03
>> MB 7:47 <https://maps.google.com/?q=MB+7:47&entry=gmail&source=g>/29.22
>> MB
>> >> >          - AverageThreadTokens: 0.00
>> >> >          - BloomFilterBytes: 0
>> >> >          - PeakMemoryUsage: 2.66 MB (2793825)
>> >> >          - PeakReservation: 0
>> >> >          - PeakUsedReservation: 0
>> >> >          - PerHostPeakMemUsage: 2.68 MB (2806113)
>> >> >          - RowsProduced: 1 (1)
>> >> >          - TotalNetworkReceiveTime: 0.000ns
>> >> >          - TotalNetworkSendTime: 198.894us
>> >> >          - TotalStorageWaitTime: 4s374ms
>> >> >          - TotalThreadsInvoluntaryContextSwitches: 110 (110)
>> >> >          - TotalThreadsTotalWallClockTime: 6s201ms
>> >> >            - TotalThreadsSysTime: 29.983ms
>> >> >            - TotalThreadsUserTime: 642.892ms
>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.17K (2174)
>> >> >         Fragment Instance Lifecycle Timings:
>> >> >            - ExecTime: 350.873us
>> >> >              - ExecTreeExecTime: 8.173us
>> >> >            - OpenTime: 275.347ms
>> >> >              - ExecTreeOpenTime: 253.425ms
>> >> >            - PrepareTime: 22.341ms
>> >> >              - ExecTreePrepareTime: 2.251ms
>> >> >         DataStreamSender (dst_id=2):(Total: 164.394us, non-child:
>> >> > 164.394us,
>> >> > % non-child: 100.00%)
>> >> >            - BytesSent: 16.00 B (16)
>> >> >            - NetworkThroughput(*): 493.42 KB/sec
>> >> >            - OverallThroughput: 95.05 KB/sec
>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - SerializeBatchTime: 14.262us
>> >> >            - TransmitDataRPCTime: 31.667us
>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>> >> >         CodeGen:(Total: 41.008ms, non-child: 41.008ms, % non-child:
>> >> > 100.00%)
>> >> >            - CodegenTime: 1.049ms
>> >> >            - CompileTime: 7.248ms
>> >> >            - LoadTime: 0.000ns
>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>> >> >            - NumFunctions: 15 (15)
>> >> >            - NumInstructions: 214 (214)
>> >> >            - OptimizationTime: 14.018ms
>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>> >> >            - PrepareTime: 19.080ms
>> >> >         AGGREGATION_NODE (id=1):(Total: 255.678ms, non-child:
>> 1.046ms, %
>> >> > non-child: 0.41%)
>> >> >           ExecOption: Codegen Enabled
>> >> >            - BuildTime: 1.015ms
>> >> >            - GetResultsTime: 0.000ns
>> >> >            - HTResizeTime: 0.000ns
>> >> >            - HashBuckets: 0 (0)
>> >> >            - LargestPartitionPercent: 0 (0)
>> >> >            - MaxPartitionLevel: 0 (0)
>> >> >            - NumRepartitions: 0 (0)
>> >> >            - PartitionsCreated: 0 (0)
>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>> >> >            - RowsRepartitioned: 0 (0)
>> >> >            - RowsReturned: 1 (1)
>> >> >            - RowsReturnedRate: 3.00 /sec
>> >> >            - SpilledPartitions: 0 (0)
>> >> >         HDFS_SCAN_NODE (id=0):(Total: 254.631ms, non-child:
>> 254.631ms, %
>> >> > non-child: 100.00%)
>> >> >           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>> >> > 9:55/35.38 MB 0:37/24.23 MB 5:37
>> <https://maps.google.com/?q=MB+5:37&entry=gmail&source=g>/23.31 MB 11:22
>> <https://maps.google.com/?q=MB+11:22&entry=gmail&source=g>/13.79 MB 8:30
>> <https://maps.google.com/?q=MB+8:30&entry=gmail&source=g>/18.53 MB
>> >> > 4:33/20.45 MB 1:38/24.06 MB 10:29
>> <https://maps.google.com/?q=MB+10:29&entry=gmail&source=g>/18.40 MB 3:39
>> <https://maps.google.com/?q=MB+3:39&entry=gmail&source=g>/24.35 MB 2:41
>> <https://maps.google.com/?q=MB+2:41&entry=gmail&source=g>/25.45 MB
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+6:42&entry=gmail&source=g>
>> >> > 6:42
>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+6:42&entry=gmail&source=g>/27.03
>> MB 7:47 <https://maps.google.com/?q=MB+7:47&entry=gmail&source=g>/29.22
>> MB
>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled: 450
>> out
>> >> > of
>> >> > 450
>> >> >           Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0%
>> 4:0%
>> >> > 5:0%
>> >> > 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>> >> >           File Formats: PARQUET/Unknown(Skipped):450
>> >> >            - FooterProcessingTime: (Avg: 10.727ms ; Min: 3.813ms ;
>> Max:
>> >> > 23.755ms ; Number of samples: 450)
>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>> >> >            - AverageScannerThreadConcurrency: 0.00
>> >> >            - BytesRead: 43.85 MB (45983284)
>> >> >            - BytesReadDataNodeCache: 0
>> >> >            - BytesReadLocal: 43.85 MB (45983284)
>> >> >            - BytesReadRemoteUnexpected: 0
>> >> >            - BytesReadShortCircuit: 43.85 MB (45983284)
>> >> >            - CachedFileHandlesHitCount: 0 (0)
>> >> >            - CachedFileHandlesMissCount: 450 (450)
>> >> >            - CollectionItemsRead: 0 (0)
>> >> >            - DecompressionTime: 0.000ns
>> >> >            - MaxCompressedTextFileLength: 0
>> >> >            - NumColumns: 0 (0)
>> >> >            - NumDictFilteredRowGroups: 0 (0)
>> >> >            - NumDisksAccessed: 12 (12)
>> >> >            - NumRowGroups: 450 (450)
>> >> >            - NumScannerThreadsStarted: 24 (24)
>> >> >            - NumScannersWithNoReads: 0 (0)
>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>> >> >            - PeakMemoryUsage: 2.62 MB (2752512)
>> >> >            - PerReadThreadRawHdfsThroughput: 1.56 GB/sec
>> >> >            - RemoteScanRanges: 0 (0)
>> >> >            - RowBatchQueueGetWaitTime: 242.362ms
>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>> >> >            - RowsRead: 0 (0)
>> >> >            - RowsReturned: 450 (450)
>> >> >            - RowsReturnedRate: 1.77 K/sec
>> >> >            - ScanRangesComplete: 450 (450)
>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 106 (106)
>> >> >            - ScannerThreadsTotalWallClockTime: 5s926ms
>> >> >              - MaterializeTupleTime(*): 69.447us
>> >> >              - ScannerThreadsSysTime: 26.984ms
>> >> >              - ScannerThreadsUserTime: 607.897ms
>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.72K (1723)
>> >> >            - TotalRawHdfsReadTime(*): 27.393ms
>> >> >            - TotalReadThroughput: 0.00 /sec
>> >> > ===================================================
>> >> >
>> >> > Coordinator log:
>> >> >
>> >> >  01: query (string) = "select count(*) from
>> dv_entersiteevent_prq_remote
>> >> > where year=2017 and month=10",
>> >> >   03: configuration (list) = list<string>[1] {
>> >> >     [0] = "REQUEST_POOL=impreports",
>> >> >   },
>> >> >   04: hadoop_user (string) = "fawzea",
>> >> > }
>> >> > I0208 01:07:06.321281  7154 impala-beeswax-server.cc:459]
>> >> > TClientRequest.queryOptions: TQueryOptions {
>> >> >   01: abort_on_error (bool) = false,
>> >> >   02: max_errors (i32) = 100,
>> >> >   03: disable_codegen (bool) = false,
>> >> >   04: batch_size (i32) = 0,
>> >> >   05: num_nodes (i32) = 0,
>> >> >   06: max_scan_range_length (i64) = 0,
>> >> >   07: num_scanner_threads (i32) = 0,
>> >> >   08: max_io_buffers (i32) = 0,
>> >> >   09: allow_unsupported_formats (bool) = false,
>> >> >   10: default_order_by_limit (i64) = -1,
>> >> >   11: debug_action (string) = "",
>> >> >   12: mem_limit (i64) = 0,
>> >> >   13: abort_on_default_limit_exceeded (bool) = false,
>> >> >   15: hbase_caching (i32) = 0,
>> >> >   16: hbase_cache_blocks (bool) = false,
>> >> >   17: parquet_file_size (i64) = 0,
>> >> >   18: explain_level (i32) = 1,
>> >> >   19: sync_ddl (bool) = false,
>> >> >   20: request_pool (string) = "impreports",
>> >> >   23: disable_cached_reads (bool) = false,
>> >> >   24: disable_outermost_topn (bool) = false,
>> >> >   25: rm_initial_mem (i64) = 0,
>> >> >   26: query_timeout_s (i32) = 0,
>> >> >   28: appx_count_distinct (bool) = false,
>> >> >   29: disable_unsafe_spills (bool) = false,
>> >> >   31: exec_single_node_rows_threshold (i32) = 100,
>> >> >   32: optimize_partition_key_scans (bool) = false,
>> >> >   33: replica_preference (i32) = 0,
>> >> >   34: schedule_random_replica (bool) = false,
>> >> >   35: scan_node_codegen_threshold (i64) = 1800000,
>> >> >   36: disable_streaming_preaggregations (bool) = false,
>> >> >   37: runtime_filter_mode (i32) = 2,
>> >> >   38: runtime_bloom_filter_size (i32) = 1048576,
>> >> >   39: runtime_filter_wait_time_ms (i32) = 0,
>> >> >   40: disable_row_runtime_filtering (bool) = false,
>> >> >   41: max_num_runtime_filters (i32) = 10,
>> >> >   42: parquet_annotate_strings_utf8 (bool) = false,
>> >> >   43: parquet_fallback_schema_resolution (i32) = 0,
>> >> >   45: s3_skip_insert_staging (bool) = true,
>> >> >   46: runtime_filter_min_size (i32) = 1048576,
>> >> >   47: runtime_filter_max_size (i32) = 16777216,
>> >> >   48: prefetch_mode (i32) = 1,
>> >> >   49: strict_mode (bool) = false,
>> >> >   50: scratch_limit (i64) = -1,
>> >> >   51: enable_expr_rewrites (bool) = true,
>> >> >   52: decimal_v2 (bool) = false,
>> >> >   53: parquet_dictionary_filtering (bool) = true,
>> >> >   54: parquet_array_resolution (i32) = 2,
>> >> >   55: parquet_read_statistics (bool) = true,
>> >> >   56: default_join_distribution_mode (i32) = 0,
>> >> >   57: disable_codegen_rows_threshold (i32) = 50000,
>> >> >   58: default_spillable_buffer_size (i64) = 2097152,
>> >> >   59: min_spillable_buffer_size (i64) = 65536,
>> >> >   60: max_row_size (i64) = 524288,
>> >> > }
>> >> > I0208 01:07:06.324410  7154 Frontend.java:909] Compiling query:
>> select
>> >> > count(*) from dv_entersiteevent_prq_remote where year=2017 and
>> month=10
>> >> > I0208 01:07:06.327203  7154 Frontend.java:948] Compiled query.
>> >> > I0208 01:07:06.359748  7154 admission-controller.cc:510] Schedule for
>> >> > id=5f49e927591a9042:faf8f28600000000 in pool_name=root.impreports
>> >> > cluster_mem_needed=105.00 MB PoolConfig: max_requests=-1
>> max_queued=200
>> >> > max_mem=-1.00 B
>> >> > I0208 01:07:06.359776  7154 admission-controller.cc:515] Stats:
>> >> > agg_num_running=0, agg_num_queued=0, agg_mem_reserved=0,
>> >> > local_host(local_mem_admitted=0, num_admitted_running=0,
>> num_queued=0,
>> >> > backend_mem_reserved=0)
>> >> > I0208 01:07:06.359784  7154 admission-controller.cc:531] Admitted
>> query
>> >> > id=5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.359807  7154 coordinator.cc:99] Exec()
>> >> > query_id=5f49e927591a9042:faf8f28600000000 stmt=select count(*) from
>> >> > dv_entersiteevent_prq_remote where year=2017 and month=10
>> >> > I0208 01:07:06.360694  7154 coordinator.cc:357] starting execution
>> on 5
>> >> > backends for query 5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.363255 63583 impala-internal-service.cc:44]
>> >> > ExecQueryFInstances(): query_id=5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.363282 63583 query-exec-mgr.cc:46]
>> StartQueryFInstances()
>> >> > query_id=5f49e927591a9042:faf8f28600000000
>> >> > coord=aopr-dhc001.lpdomain.com:22000
>> >> > I0208 01:07:06.363296 63583 query-state.cc:173] Buffer pool limit for
>> >> > 5f49e927591a9042:faf8f28600000000: 10802010521
>> >> > I0208 01:07:06.363364 63583 initial-reservations.cc:60] Successfully
>> >> > claimed
>> >> > initial reservations (0) for query 5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.363544  7155 query-state.cc:286] StartFInstances():
>> >> > query_id=5f49e927591a9042:faf8f28600000000 #instances=2
>> >> > I0208 01:07:06.363844  7155 query-state.cc:299] descriptor table for
>> >> > query=5f49e927591a9042:faf8f28600000000
>> >> > tuples:
>> >> > Tuple(id=2 size=0 slots=[] tuple_path=[])
>> >> > Tuple(id=1 size=8 slots=[Slot(id=2 type=BIGINT col_path=[] offset=0
>> >> > null=(offset=0 mask=0) slot_idx=0 field_idx=-1)] tuple_path=[])
>> >> > Tuple(id=0 size=8 slots=[Slot(id=3 type=BIGINT col_path=[] offset=0
>> >> > null=(offset=0 mask=0) slot_idx=0 field_idx=-1)] tuple_path=[])
>> >> > I0208 01:07:06.364013  7156 query-state.cc:377] Executing instance.
>> >> > instance_id=5f49e927591a9042:faf8f28600000000 fragment_idx=0
>> >> > per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=1
>> >> > I0208 01:07:06.364135  7157 query-state.cc:377] Executing instance.
>> >> > instance_id=5f49e927591a9042:faf8f28600000005 fragment_idx=1
>> >> > per_fragment_instance_idx=4 coord_state_idx=0 #in-flight=2
>> >> > I0208 01:07:06.364241  7157 hdfs-scan-node.cc:160] Max row batch
>> queue
>> >> > size
>> >> > for scan node '0' in fragment instance
>> >> > '5f49e927591a9042:faf8f28600000005':
>> >> > 160
>> >> > I0208 01:07:06.366775  7154 coordinator.cc:370] started execution on
>> 5
>> >> > backends for query 5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.385625  7155 query-exec-mgr.cc:149]
>> ReleaseQueryState():
>> >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=4
>> >> > I0208 01:07:06.671193  7559 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.16.136.3:55598 closed, closing 1 associated session(s)
>> >> > I0208 01:07:06.676138   544 data-stream-mgr.cc:228] Reduced stream ID
>> >> > cache
>> >> > from 2 items, to 0, eviction took: 0
>> >> > I0208 01:07:06.677202   544 coordinator.cc:955] Backend completed:
>> >> > host=aopr-dhc002.lpdomain.com:22000 remaining=4
>> >> > I0208 01:07:06.677214   544 coordinator-backend-state.cc:229]
>> >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress
>> backend:
>> >> > aopr-dhc001.lpdomain.com:22000
>> >> > I0208 01:07:06.684221  7157 query-state.cc:384] Instance completed.
>> >> > instance_id=5f49e927591a9042:faf8f28600000005 #in-flight=1 status=OK
>> >> > I0208 01:07:06.684237  7157 query-exec-mgr.cc:149]
>> ReleaseQueryState():
>> >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=3
>> >> > I0208 01:07:06.700275   520 coordinator.cc:955] Backend completed:
>> >> > host=aopr-dhc005.lpdomain.com:22000 remaining=3
>> >> > I0208 01:07:06.700289   520 coordinator-backend-state.cc:229]
>> >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress
>> backend:
>> >> > aopr-dhc001.lpdomain.com:22000
>> >> > I0208 01:07:06.704620 65258 coordinator.cc:955] Backend completed:
>> >> > host=aopr-dhc003.lpdomain.com:22000 remaining=2
>> >> > I0208 01:07:06.704666 65258 coordinator-backend-state.cc:229]
>> >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress
>> backend:
>> >> > aopr-dhc001.lpdomain.com:22000
>> >> > I0208 01:07:06.706041  7156 data-stream-mgr.cc:238]
>> DeregisterRecvr():
>> >> > fragment_instance_id=5f49e927591a9042:faf8f28600000000, node=2
>> >> > I0208 01:07:06.706070  7156 data-stream-recvr.cc:235] cancelled
>> stream:
>> >> > fragment_instance_id_=5f49e927591a9042:faf8f28600000000 node_id=2
>> >> > I0208 01:07:06.706809 64916 coordinator.cc:955] Backend completed:
>> >> > host=aopr-dhc004.lpdomain.com:22000 remaining=1
>> >> > I0208 01:07:06.706821 64916 coordinator-backend-state.cc:229]
>> >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress
>> backend:
>> >> > aopr-dhc001.lpdomain.com:22000
>> >> > I0208 01:07:06.789568  7154 impala-beeswax-server.cc:193]
>> >> > get_results_metadata(): query_id=5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.790439  7154 coordinator.cc:789] Coordinator waiting
>> for
>> >> > backends to finish, 1 remaining
>> >> > I0208 01:07:06.791052  7154 coordinator.cc:794] All backends finished
>> >> > successfully.
>> >> > I0208 01:07:06.791086  7154 coordinator.cc:1090] Release admssion
>> >> > control
>> >> > resources for query 5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.791363  7156 query-state.cc:384] Instance completed.
>> >> > instance_id=5f49e927591a9042:faf8f28600000000 #in-flight=0 status=OK
>> >> > I0208 01:07:06.791414  7156 query-exec-mgr.cc:149]
>> ReleaseQueryState():
>> >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=2
>> >> > I0208 01:07:06.793586  7154 impala-beeswax-server.cc:235] close():
>> >> > query_id=5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.793602  7154 impala-server.cc:992] UnregisterQuery():
>> >> > query_id=5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.793614  7154 impala-server.cc:1075] Cancel():
>> >> > query_id=5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.793624  7154 coordinator.cc:895] Cancel()
>> >> > query_id=5f49e927591a9042:faf8f28600000000
>> >> > I0208 01:07:06.793637  7154 coordinator.cc:905] CancelBackends()
>> >> > query_id=5f49e927591a9042:faf8f28600000000, tried to cancel 0
>> backends
>> >> > I0208 01:07:06.802776  7154 query-exec-mgr.cc:149]
>> ReleaseQueryState():
>> >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=1
>> >> > I0208 01:07:06.814214  7154 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.16.144.131:51012 closed, closing 1 associated session(s)
>> >> > I0208 01:07:07.258891  7567 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.16.136.2:56375 closed, closing 1 associated session(s)
>> >> > I0208 01:07:11.600131  7819 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.16.136.3:27460 closed, closing 1 associated session(s)
>> >> > I0208 01:07:12.262739  7821 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.16.136.2:52672 closed, closing 1 associated session(s)
>> >> > I0208 01:07:16.665451  7822 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.16.136.3:10693 closed, closing 1 associated session(s)
>> >> > I0208 01:07:17.235321  7825 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.16.136.2:34752 closed, closing 1 associated session(s)
>> >> > I0208 01:07:21.410611  5858 impala-beeswax-server.cc:193]
>> >> > get_results_metadata(): query_id=1d4fbf0f2571f1e9:ea346f7900000000
>> >> > I0208 01:07:21.414175  5858 impala-beeswax-server.cc:235] close():
>> >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000
>> >> > I0208 01:07:21.414185  5858 impala-server.cc:992] UnregisterQuery():
>> >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000
>> >> > I0208 01:07:21.414191  5858 impala-server.cc:1075] Cancel():
>> >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000
>> >> > I0208 01:07:21.421505  5858 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.26.180.75:60072 closed, closing 1 associated session(s)
>> >> > I0208 01:07:21.605690  7830 impala-server.cc:1796] Connection from
>> >> > client
>> >> > ::ffff:10.16.136.3:46660 closed, closing 1 associated session(s)
>> >> >
>> >> >
>> >> > On Thu, Feb 8, 2018 at 6:20 AM, Matthew Jacobs <mjacobs@apache.org>
>> >> > wrote:
>> >> >>
>> >> >> Thanks! I think that configuration looks OK.
>> >> >>
>> >> >> 1) What version of Impala is this?
>> >> >>
>> >> >> 2) Can you send the query profile and the Impala coordinator log for
>> >> >> the query where it's not behaving as expected?
>> >> >>
>> >> >> > When i'm running the following with fawzea user:
>> >> >> > set request_pool=impreports;
>> >> >> > select count(*) from dv_entersiteevent_prq_remote where year=2017
>> and
>> >> >> > month=10;
>> >> >> > I can see that the query is running with impreports pool.
>> >> >>
>> >> >> On Wed, Feb 7, 2018 at 7:59 PM, Fawze Abujaber <fawzeaj@gmail.com>
>> >> >> wrote:
>> >> >> > Hi Matt,
>> >> >> >
>> >> >> > [root@aopr-dhc001 ~]# cat
>> >> >> >
>> >> >> >
>> >> >> > /var/run/cloudera-scm-agent/process/4558-impala-IMPALAD/impa
>> la-conf/fair-scheduler.xml
>> >> >> > <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
>> >> >> > <allocations>
>> >> >> >     <queue name="root">
>> >> >> >         <weight>1.0</weight>
>> >> >> >         <schedulingPolicy>fair</schedulingPolicy>
>> >> >> >         <queue name="impiaas">
>> >> >> >             <weight>1.0</weight>
>> >> >> >             <schedulingPolicy>fair</schedulingPolicy>
>> >> >> >             <aclSubmitApps>fawzea </aclSubmitApps>
>> >> >> >         </queue>
>> >> >> >         <queue name="impreports">
>> >> >> >             <weight>1.0</weight>
>> >> >> >             <schedulingPolicy>fair</schedulingPolicy>
>> >> >> >             <aclSubmitApps>nira </aclSubmitApps>
>> >> >> >         </queue>
>> >> >> >     </queue>
>> >> >> >     <defaultQueueSchedulingPolicy>fair</defaultQueueSchedulingPol
>> icy>
>> >> >> >     <queuePlacementPolicy>
>> >> >> >         <rule name="specified" create="false"/>
>> >> >> >         <rule name="secondaryGroupExistingQueue"/>
>> >> >> >         <rule name="default"/>
>> >> >> >     </queuePlacementPolicy>
>> >> >> > </allocations>
>> >> >> >
>> >> >> >
>> >> >> > On Thu, Feb 8, 2018 at 4:47 AM, Matthew Jacobs <
>> jacobs.mh@gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Yes, I saw that, but I'd like to see the actual configuration
>> that
>> >> >> >> is
>> >> >> >> passed to Impala because it may not be specified in the right
>> way to
>> >> >> >> accomplish what you're trying to do in "b".
>> >> >> >>
>> >> >> >> On Wed, Feb 7, 2018 at 4:32 PM, Fawze Abujaber <
>> fawzeaj@gmail.com>
>> >> >> >> wrote:
>> >> >> >> > Hi Matt,
>> >> >> >> >
>> >> >> >> > I created the pools manually and it’s completely nfigurations
>> as
>> >> >> >> > shown
>> >> >> >> > in b.
>> >> >> >> >
>> >> >> >> > On Wed, 7 Feb 2018 at 23:17 Matthew Jacobs <
>> jacobs.mh@gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> How, more specifically, are you configuring these pools? Can
>> you
>> >> >> >> >> share
>> >> >> >> >> the configuration files?
>> >> >> >> >>
>> >> >> >> >> On Wed, Feb 7, 2018 at 11:42 AM, Fawze Abujaber
>> >> >> >> >> <fawzeaj@gmail.com>
>> >> >> >> >> wrote:
>> >> >> >> >> > Hello All,
>> >> >> >> >> >
>> >> >> >> >> > Will appreciate any insight of what i'm missing in my
>> >> >> >> >> > configuration
>> >> >> >> >> > to
>> >> >> >> >> > get
>> >> >> >> >> > the impala submission control working as expected ....
>> >> >> >> >> >
>> >> >> >> >> > a- Here is my placement Placement Rule:
>> >> >> >> >> > 1- Use the pool Specified at run time., only if the pool
>> >> >> >> >> > exists.
>> >> >> >> >> > 2- Use the pool root.[secondary group].
>> >> >> >> >> > 3- Use the pool root.default.
>> >> >> >> >> >
>> >> >> >> >> > b- Here is the impala resource pools
>> >> >> >> >> >
>> >> >> >> >> > Name                    Max Memory Max Running Queries Max
>> >> >> >> >> > Queued
>> >> >> >> >> > Queries
>> >> >> >> >> > Queue Timeout Default Query Memory Limit Access Control
>> >> >> >> >> > root.impiaas            Unlimited
>>  Unlimited
>> >> >> >> >> > 200                          1 minute(s)            No
>> Default
>> >> >> >> >> > Custom
>> >> >> >> >> > root.impreports        Unlimited                   Unlimited
>> >> >> >> >> > 200                          1 minute(s)            No
>> Default
>> >> >> >> >> > Custom
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > C- Here is my Access control for root.impiaas pool:
>> >> >> >> >> >
>> >> >> >> >> > Allow these users and groups to submit to this pool
>> >> >> >> >> >
>> >> >> >> >> > Users: fawzea
>> >> >> >> >> >
>> >> >> >> >> > The same for impreports the allowed user and groups to
>> submit
>> >> >> >> >> > to
>> >> >> >> >> > pool
>> >> >> >> >> > is
>> >> >> >> >> > only nira.
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > LDAP groups for fawzea user:
>> >> >> >> >> >
>> >> >> >> >> >  Member in groups:
>> >> >> >> >> >
>> >> >> >> >> >  - dba
>> >> >> >> >> >  - impiaas
>> >> >> >> >> >  - mysql
>> >> >> >> >> >
>> >> >> >> >> >  LDAP group for nira user
>> >> >> >> >> >   Member in groups:
>> >> >> >> >> >  - dba
>> >> >> >> >> >  - impreports
>> >> >> >> >> >  - mysql
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > Impala Queries UI shows:
>> >> >> >> >> >
>> >> >> >> >> > 1- select count(*) from dv_entersiteevent_prq_remote where
>> >> >> >> >> > year=2017
>> >> >> >> >> >
>> >> >> >> >> > User: fawzea
>> >> >> >> >> > Pool: root.impiaas
>> >> >> >> >> >
>> >> >> >> >> > 2- select count(*) from dv_entersiteevent_prq_remote where
>> >> >> >> >> > year=2017
>> >> >> >> >> > and
>> >> >> >> >> > month=10
>> >> >> >> >> > User: nira
>> >> >> >> >> > Pool: root.impreports
>> >> >> >> >> >
>> >> >> >> >> > As shown above that the queries submitted based on the
>> >> >> >> >> > secondary
>> >> >> >> >> > group
>> >> >> >> >> > rule.
>> >> >> >> >> >
>> >> >> >> >> > When i'm running the following with fawzea user:
>> >> >> >> >> >
>> >> >> >> >> > set request_pool=impreports;
>> >> >> >> >> > select count(*) from dv_entersiteevent_prq_remote where
>> >> >> >> >> > year=2017
>> >> >> >> >> > and
>> >> >> >> >> > month=10;
>> >> >> >> >> >
>> >> >> >> >> > I can see that the query is running with impreports pool.
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > What i'm missing here in my configuration to get this works
>> as
>> >> >> >> >> > expected.
>> >> >> >> >> >
>> >> >> >
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>>
>
>

Mime
View raw message