impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fawze Abujaber <fawz...@gmail.com>
Subject Re: impala submission control
Date Thu, 08 Feb 2018 21:03:41 GMT
Sorry, but i'm able to replicate the issue on impala
version:  2.10.0-cdh5.13.0

I will be happy if anyone who can try to replicate the issue on his test
cluster.

On Thu, Feb 8, 2018 at 8:25 PM, Matthew Jacobs <jacobs.mh@gmail.com> wrote:

> I just filed one: https://issues.apache.org/jira/browse/IMPALA-6493
>
> On Thu, Feb 8, 2018 at 10:12 AM, Matthew Jacobs <jacobs.mh@gmail.com>
> wrote:
>
>> As I said, I don't see anything wrong with the configuration so there
>> could be a bug in Impala 2.11, likely due to IMPALA-5920. Can you file a
>> JIRA?
>>
>> On Thu, Feb 8, 2018 at 9:42 AM, Fawze Abujaber <fawzeaj@gmail.com> wrote:
>>
>>> This is not fitting my situation,
>>>
>>> I removed even the placement role of the secondary group, and other
>>> users can still submit to the pool that i restrict it to my user.
>>>
>>> Just thinking in loud and think the problem might be in using the
>>> placement rule:
>>>
>>> Use the pool Specified at run time., only if the pool exists.
>>> Edit
>>>
>>>
>>> as it it the first rule, it always successes.
>>>
>>>
>>> On Thu, Feb 8, 2018 at 7:12 PM, Matthew Jacobs <jacobs.mh@gmail.com>
>>> wrote:
>>>
>>>> The aclSubmitApps can specify users or groups, so if the user name is
>>>> also a group name, that can lead to confusion, e.g.
>>>> https://issues.apache.org/jira/browse/IMPALA-2578
>>>> I'm not saying that is necessarily the problem, but something to
>>>> check. If we still can't find anything configured incorrectly I'd
>>>> recommend that you file a JIRA.
>>>>
>>>> On Thu, Feb 8, 2018 at 9:05 AM, Fawze Abujaber <fawzeaj@gmail.com>
>>>> wrote:
>>>> > Hi Matt,
>>>> >
>>>> > If this is the situation, how we can explain that when i submit the
>>>> query it
>>>> > runs on my secondary group pool and when nira do the same it runs
>>>> under her
>>>> > secondary group pool.
>>>> >
>>>> > More than this, the issue not related to LDAP group as i'm using the
>>>> > submission restriction to a user and not a group.
>>>> >
>>>> > On Thu, Feb 8, 2018 at 7:00 PM, Matthew Jacobs <jacobs.mh@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> As far as I can tell from this information, your configuration looks
>>>> >> fine. The only other potential issue I can think of is that the
>>>> >> secondary groups placement rule isn't picking up your LDAP groups
>>>> >> properly. Does the ACL behavior work if you remove the secondary
>>>> >> groups rule?
>>>> >>
>>>> >> On Wed, Feb 7, 2018 at 10:09 PM, Fawze Abujaber <fawzeaj@gmail.com>
>>>> wrote:
>>>> >> > Hi Matt,
>>>> >> >
>>>> >> > v2.11.0-cdh5.14.0
>>>> >> >
>>>> >> > Query Runtime Profile:
>>>> >> > Query (id=c843dae48ce9b289:6ad3ffc000000000):
>>>> >> >   Summary:
>>>> >> >     Session ID: ca420ac477772b56:51ae460ce9e44999
>>>> >> >     Session Type: BEESWAX
>>>> >> >     Start Time: 2018-02-08 01:02:01.023558000
>>>> >> >     End Time: 2018-02-08 01:02:01.501506000
>>>> >> >     Query Type: QUERY
>>>> >> >     Query State: FINISHED
>>>> >> >     Query Status: OK
>>>> >> >     Impala Version: impalad version 2.11.0-cdh5.14.0 RELEASE (build
>>>> >> > d68206561bce6b26762d62c01a78e6cd27aa7690)
>>>> >> >     User: fawzea
>>>> >> >     Connected User: fawzea
>>>> >> >     Delegated User:
>>>> >> >     Network Address: ::ffff:10.16.144.131:50078
>>>> >> >     Default Db: default
>>>> >> >     Sql Statement: select count(*) from
>>>> dv_entersiteevent_prq_remote
>>>> >> > where
>>>> >> > year=2017 and month=10
>>>> >> >     Coordinator: aopr-dhc001.lpdomain.com:22000
>>>> >> >     Query Options (set by configuration): REQUEST_POOL=impreports
>>>> >> >     Query Options (set by configuration and planner):
>>>> >> > REQUEST_POOL=impreports,MT_DOP=0
>>>> >> >     Plan:
>>>> >> > ----------------
>>>> >> > Max Per-Host Resource Reservation: Memory=0B
>>>> >> > Per-Host Resource Estimates: Memory=21.00MB
>>>> >> > WARNING: The following tables are missing relevant table and/or
>>>> column
>>>> >> > statistics.
>>>> >> > default.dv_entersiteevent_prq_remote
>>>> >> >
>>>> >> > F01:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
>>>> >> > |  Per-Host Resources: mem-estimate=10.00MB mem-reservation=0B
>>>> >> > PLAN-ROOT SINK
>>>> >> > |  mem-estimate=0B mem-reservation=0B
>>>> >> > |
>>>> >> > 03:AGGREGATE [FINALIZE]
>>>> >> > |  output: count:merge(*)
>>>> >> > |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
>>>> >> > |  tuple-ids=1 row-size=8B cardinality=1
>>>> >> > |
>>>> >> > 02:EXCHANGE [UNPARTITIONED]
>>>> >> > |  mem-estimate=0B mem-reservation=0B
>>>> >> > |  tuple-ids=1 row-size=8B cardinality=1
>>>> >> > |
>>>> >> > F00:PLAN FRAGMENT [RANDOM] hosts=5 instances=5
>>>> >> > Per-Host Resources: mem-estimate=11.00MB mem-reservation=0B
>>>> >> > 01:AGGREGATE
>>>> >> > |  output:
>>>> >> > sum_init_zero(default.dv_entersiteevent_prq_remote.parquet-stats:
>>>> >> > num_rows)
>>>> >> > |  mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB
>>>> >> > |  tuple-ids=1 row-size=8B cardinality=1
>>>> >> > |
>>>> >> > 00:SCAN HDFS [default.dv_entersiteevent_prq_remote, RANDOM]
>>>> >> >    partitions=31/441 files=2292 size=1.39GB
>>>> >> >    stats-rows=unavailable extrapolated-rows=disabled
>>>> >> >    table stats: rows=unavailable size=unavailable
>>>> >> >    column stats: all
>>>> >> >    mem-estimate=1.00MB mem-reservation=0B
>>>> >> >    tuple-ids=0 row-size=8B cardinality=unavailable
>>>> >> > ----------------
>>>> >> >     Estimated Per-Host Mem: 22020096
>>>> >> >     Tables Missing Stats: default.dv_entersiteevent_prq_remote
>>>> >> >     Per Host Min Reservation: aopr-dhc001.lpdomain.com:22000(0)
>>>> >> > aopr-dhc002.lpdomain.com:22000(0) aopr-dhc003.lpdomain.com:22000
>>>> (0)
>>>> >> > aopr-dhc004.lpdomain.com:22000(0) aopr-dhc005.lpdomain.com:22000
>>>> (0)
>>>> >> >     Request Pool: root.impreports
>>>> >> >     Admission result: Admitted immediately
>>>> >> >     ExecSummary:
>>>> >> > Operator       #Hosts   Avg Time   Max Time  #Rows  Est. #Rows
>>>> Peak Mem
>>>> >> > Est. Peak Mem  Detail
>>>> >> >
>>>> >> > ------------------------------------------------------------
>>>> ------------------------------------------------------------------
>>>> >> > 03:AGGREGATE        1  222.958us  222.958us      1           1
>>>> 36.00 KB
>>>> >> > 10.00 MB  FINALIZE
>>>> >> > 02:EXCHANGE         1  357.508ms  357.508ms      5           1
>>>>      0
>>>> >> > 0  UNPARTITIONED
>>>> >> > 01:AGGREGATE        5    1.027ms    1.541ms      5           1
>>>> 20.00 KB
>>>> >> > 10.00 MB
>>>> >> > 00:SCAN HDFS        5  282.797ms  333.715ms  2.29K          -1
>>>>  2.62 MB
>>>> >> > 1.00 MB  default.dv_entersiteevent_prq_remote
>>>> >> >     Errors:
>>>> >> >     Planner Timeline: 13.508ms
>>>> >> >        - Analysis finished: 3.303ms (3.303ms)
>>>> >> >        - Value transfer graph computed: 3.465ms (162.667us)
>>>> >> >        - Single node plan created: 10.352ms (6.886ms)
>>>> >> >        - Runtime filters computed: 10.677ms (325.364us)
>>>> >> >        - Distributed plan created: 10.733ms (55.854us)
>>>> >> >        - Lineage info computed: 10.898ms (164.937us)
>>>> >> >        - Planning finished: 13.508ms (2.610ms)
>>>> >> >     Query Timeline: 481.950ms
>>>> >> >        - Query submitted: 79.971us (79.971us)
>>>> >> >        - Planning finished: 33.947ms (33.867ms)
>>>> >> >        - Submit for admission: 40.067ms (6.120ms)
>>>> >> >        - Completed admission: 40.197ms (130.146us)
>>>> >> >        - Ready to start on 5 backends: 40.905ms (707.852us)
>>>> >> >        - All 5 execution backends (6 fragment instances) started:
>>>> >> > 46.417ms
>>>> >> > (5.512ms)
>>>> >> >        - Rows available: 424.548ms (378.131ms)
>>>> >> >        - First row fetched: 473.488ms (48.939ms)
>>>> >> >        - Unregister query: 477.951ms (4.462ms)
>>>> >> >      - ComputeScanRangeAssignmentTimer: 3.980ms
>>>> >> >   ImpalaServer:
>>>> >> >      - ClientFetchWaitTimer: 51.659ms
>>>> >> >      - RowMaterializationTimer: 1.726ms
>>>> >> >   Execution Profile c843dae48ce9b289:6ad3ffc000000000:(Total:
>>>> 385.029ms,
>>>> >> > non-child: 0.000ns, % non-child: 0.00%)
>>>> >> >     Number of filters: 0
>>>> >> >     Filter routing table:
>>>> >> >  ID  Src. Node  Tgt. Node(s)  Target type  Partition filter
>>>> Pending
>>>> >> > (Expected)  First arrived  Completed   Enabled
>>>> >> >
>>>> >> > ------------------------------------------------------------
>>>> -------------------------------------------------------
>>>> >> >
>>>> >> >     Backend startup latencies: Count: 5, min / max: 3ms / 4ms, 25th
>>>> >> > %-ile:
>>>> >> > 3ms, 50th %-ile: 4ms, 75th %-ile: 4ms, 90th %-ile: 4ms, 95th
>>>> %-ile: 4ms,
>>>> >> > 99.9th %-ile: 4ms
>>>> >> >     Per Node Peak Memory Usage: aopr-dhc001.lpdomain.com:22000(2.68
>>>> MB)
>>>> >> > aopr-dhc002.lpdomain.com:22000(2.54 MB)
>>>> >> > aopr-dhc003.lpdomain.com:22000(2.54
>>>> >> > MB) aopr-dhc004.lpdomain.com:22000(2.41 MB)
>>>> >> > aopr-dhc005.lpdomain.com:22000(2.56 MB)
>>>> >> >      - FiltersReceived: 0 (0)
>>>> >> >      - FinalizationTimer: 0.000ns
>>>> >> >      - NumBackends: 5 (5)
>>>> >> >      - NumFragmentInstances: 6 (6)
>>>> >> >      - NumFragments: 2 (2)
>>>> >> >     Averaged Fragment F01:(Total: 427.530ms, non-child: 49.428ms, %
>>>> >> > non-child: 11.56%)
>>>> >> >       split sizes:  min: 0, max: 0, avg: 0, stddev: 0
>>>> >> >       completion times: min:428.693ms  max:428.693ms  mean:
>>>> 428.693ms
>>>> >> > stddev:0.000ns
>>>> >> >       execution rates: min:0.00 /sec  max:0.00 /sec  mean:0.00 /sec
>>>> >> > stddev:0.00 /sec
>>>> >> >       num instances: 1
>>>> >> >        - AverageThreadTokens: 0.00
>>>> >> >        - BloomFilterBytes: 0
>>>> >> >        - PeakMemoryUsage: 44.02 KB (45072)
>>>> >> >        - PeakReservation: 0
>>>> >> >        - PeakUsedReservation: 0
>>>> >> >        - PerHostPeakMemUsage: 2.68 MB (2806113)
>>>> >> >        - RowsProduced: 1 (1)
>>>> >> >        - TotalNetworkReceiveTime: 357.481ms
>>>> >> >        - TotalNetworkSendTime: 0.000ns
>>>> >> >        - TotalStorageWaitTime: 0.000ns
>>>> >> >        - TotalThreadsInvoluntaryContextSwitches: 0 (0)
>>>> >> >        - TotalThreadsTotalWallClockTime: 406.708ms
>>>> >> >          - TotalThreadsSysTime: 0.000ns
>>>> >> >          - TotalThreadsUserTime: 0.000ns
>>>> >> >        - TotalThreadsVoluntaryContextSwitches: 7 (7)
>>>> >> >       Fragment Instance Lifecycle Timings:
>>>> >> >          - ExecTime: 49.055ms
>>>> >> >            - ExecTreeExecTime: 4.251us
>>>> >> >          - OpenTime: 357.663ms
>>>> >> >            - ExecTreeOpenTime: 357.649ms
>>>> >> >          - PrepareTime: 20.764ms
>>>> >> >            - ExecTreePrepareTime: 81.759us
>>>> >> >       PLAN_ROOT_SINK:
>>>> >> >          - PeakMemoryUsage: 0
>>>> >> >       CodeGen:(Total: 20.369ms, non-child: 20.369ms, % non-child:
>>>> >> > 100.00%)
>>>> >> >          - CodegenTime: 0.000ns
>>>> >> >          - CompileTime: 0.000ns
>>>> >> >          - LoadTime: 0.000ns
>>>> >> >          - ModuleBitcodeSize: 1.86 MB (1950880)
>>>> >> >          - NumFunctions: 0 (0)
>>>> >> >          - NumInstructions: 0 (0)
>>>> >> >          - OptimizationTime: 0.000ns
>>>> >> >          - PeakMemoryUsage: 0
>>>> >> >          - PrepareTime: 19.685ms
>>>> >> >       AGGREGATION_NODE (id=3):(Total: 357.731ms, non-child:
>>>> 222.958us, %
>>>> >> > non-child: 0.06%)
>>>> >> >          - BuildTime: 7.368us
>>>> >> >          - GetResultsTime: 0.000ns
>>>> >> >          - HTResizeTime: 0.000ns
>>>> >> >          - HashBuckets: 0 (0)
>>>> >> >          - LargestPartitionPercent: 0 (0)
>>>> >> >          - MaxPartitionLevel: 0 (0)
>>>> >> >          - NumRepartitions: 0 (0)
>>>> >> >          - PartitionsCreated: 0 (0)
>>>> >> >          - PeakMemoryUsage: 36.00 KB (36864)
>>>> >> >          - RowsRepartitioned: 0 (0)
>>>> >> >          - RowsReturned: 1 (1)
>>>> >> >          - RowsReturnedRate: 2.00 /sec
>>>> >> >          - SpilledPartitions: 0 (0)
>>>> >> >       EXCHANGE_NODE (id=2):(Total: 357.508ms, non-child:
>>>> 357.508ms, %
>>>> >> > non-child: 100.00%)
>>>> >> >          - ConvertRowBatchTime: 8.881us
>>>> >> >          - PeakMemoryUsage: 0
>>>> >> >          - RowsReturned: 5 (5)
>>>> >> >          - RowsReturnedRate: 13.00 /sec
>>>> >> >         DataStreamReceiver:
>>>> >> >            - BytesReceived: 80.00 B (80)
>>>> >> >            - DeserializeRowBatchTimer: 24.165us
>>>> >> >            - FirstBatchArrivalWaitTime: 277.194ms
>>>> >> >            - PeakMemoryUsage: 4.02 KB (4112)
>>>> >> >            - SendersBlockedTimer: 0.000ns
>>>> >> >            - SendersBlockedTotalTimer(*): 0.000ns
>>>> >> >     Coordinator Fragment F01:
>>>> >> >       Instance c843dae48ce9b289:6ad3ffc000000000
>>>> >> > (host=aopr-dhc001.lpdomain.com:22000):(Total: 427.530ms,
>>>> non-child:
>>>> >> > 49.428ms, % non-child: 11.56%)
>>>> >> >          - AverageThreadTokens: 0.00
>>>> >> >          - BloomFilterBytes: 0
>>>> >> >          - PeakMemoryUsage: 44.02 KB (45072)
>>>> >> >          - PeakReservation: 0
>>>> >> >          - PeakUsedReservation: 0
>>>> >> >          - PerHostPeakMemUsage: 2.68 MB (2806113)
>>>> >> >          - RowsProduced: 1 (1)
>>>> >> >          - TotalNetworkReceiveTime: 357.481ms
>>>> >> >          - TotalNetworkSendTime: 0.000ns
>>>> >> >          - TotalStorageWaitTime: 0.000ns
>>>> >> >          - TotalThreadsInvoluntaryContextSwitches: 0 (0)
>>>> >> >          - TotalThreadsTotalWallClockTime: 406.708ms
>>>> >> >            - TotalThreadsSysTime: 0.000ns
>>>> >> >            - TotalThreadsUserTime: 0.000ns
>>>> >> >          - TotalThreadsVoluntaryContextSwitches: 7 (7)
>>>> >> >         Fragment Instance Lifecycle Timings:
>>>> >> >            - ExecTime: 49.055ms
>>>> >> >              - ExecTreeExecTime: 4.251us
>>>> >> >            - OpenTime: 357.663ms
>>>> >> >              - ExecTreeOpenTime: 357.649ms
>>>> >> >            - PrepareTime: 20.764ms
>>>> >> >              - ExecTreePrepareTime: 81.759us
>>>> >> >         PLAN_ROOT_SINK:
>>>> >> >            - PeakMemoryUsage: 0
>>>> >> >         CodeGen:(Total: 20.369ms, non-child: 20.369ms, % non-child:
>>>> >> > 100.00%)
>>>> >> >            - CodegenTime: 0.000ns
>>>> >> >            - CompileTime: 0.000ns
>>>> >> >            - LoadTime: 0.000ns
>>>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>>>> >> >            - NumFunctions: 0 (0)
>>>> >> >            - NumInstructions: 0 (0)
>>>> >> >            - OptimizationTime: 0.000ns
>>>> >> >            - PeakMemoryUsage: 0
>>>> >> >            - PrepareTime: 19.685ms
>>>> >> >         AGGREGATION_NODE (id=3):(Total: 357.731ms, non-child:
>>>> 222.958us,
>>>> >> > %
>>>> >> > non-child: 0.06%)
>>>> >> >            - BuildTime: 7.368us
>>>> >> >            - GetResultsTime: 0.000ns
>>>> >> >            - HTResizeTime: 0.000ns
>>>> >> >            - HashBuckets: 0 (0)
>>>> >> >            - LargestPartitionPercent: 0 (0)
>>>> >> >            - MaxPartitionLevel: 0 (0)
>>>> >> >            - NumRepartitions: 0 (0)
>>>> >> >            - PartitionsCreated: 0 (0)
>>>> >> >            - PeakMemoryUsage: 36.00 KB (36864)
>>>> >> >            - RowsRepartitioned: 0 (0)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - RowsReturnedRate: 2.00 /sec
>>>> >> >            - SpilledPartitions: 0 (0)
>>>> >> >         EXCHANGE_NODE (id=2):(Total: 357.508ms, non-child:
>>>> 357.508ms, %
>>>> >> > non-child: 100.00%)
>>>> >> >            - ConvertRowBatchTime: 8.881us
>>>> >> >            - PeakMemoryUsage: 0
>>>> >> >            - RowsReturned: 5 (5)
>>>> >> >            - RowsReturnedRate: 13.00 /sec
>>>> >> >           DataStreamReceiver:
>>>> >> >              - BytesReceived: 80.00 B (80)
>>>> >> >              - DeserializeRowBatchTimer: 24.165us
>>>> >> >              - FirstBatchArrivalWaitTime: 277.194ms
>>>> >> >              - PeakMemoryUsage: 4.02 KB (4112)
>>>> >> >              - SendersBlockedTimer: 0.000ns
>>>> >> >              - SendersBlockedTotalTimer(*): 0.000ns
>>>> >> >     Averaged Fragment F00:(Total: 327.360ms, non-child: 1.578ms, %
>>>> >> > non-child: 0.48%)
>>>> >> >       split sizes:  min: 283.31 MB, max: 284.33 MB, avg: 284.04 MB,
>>>> >> > stddev:
>>>> >> > 382.00 KB
>>>> >> >       completion times: min:299.457ms  max:378.950ms  mean:
>>>> 328.881ms
>>>> >> > stddev:31.507ms
>>>> >> >       execution rates: min:750.18 MB/sec  max:949.01 MB/sec
>>>> mean:871.27
>>>> >> > MB/sec  stddev:79.74 MB/sec
>>>> >> >       num instances: 5
>>>> >> >        - AverageThreadTokens: 15.00
>>>> >> >        - BloomFilterBytes: 0
>>>> >> >        - PeakMemoryUsage: 2.54 MB (2666852)
>>>> >> >        - PeakReservation: 0
>>>> >> >        - PeakUsedReservation: 0
>>>> >> >        - PerHostPeakMemUsage: 2.55 MB (2669309)
>>>> >> >        - RowsProduced: 1 (1)
>>>> >> >        - TotalNetworkReceiveTime: 0.000ns
>>>> >> >        - TotalNetworkSendTime: 303.541us
>>>> >> >        - TotalStorageWaitTime: 4s821ms
>>>> >> >        - TotalThreadsInvoluntaryContextSwitches: 104 (104)
>>>> >> >        - TotalThreadsTotalWallClockTime: 6s835ms
>>>> >> >          - TotalThreadsSysTime: 31.784ms
>>>> >> >          - TotalThreadsUserTime: 682.084ms
>>>> >> >        - TotalThreadsVoluntaryContextSwitches: 2.12K (2121)
>>>> >> >       Fragment Instance Lifecycle Timings:
>>>> >> >          - ExecTime: 500.816us
>>>> >> >            - ExecTreeExecTime: 8.200us
>>>> >> >          - OpenTime: 302.848ms
>>>> >> >            - ExecTreeOpenTime: 281.309ms
>>>> >> >          - PrepareTime: 23.946ms
>>>> >> >            - ExecTreePrepareTime: 2.513ms
>>>> >> >       DataStreamSender (dst_id=2):(Total: 269.902us, non-child:
>>>> >> > 269.902us, %
>>>> >> > non-child: 100.00%)
>>>> >> >          - BytesSent: 16.00 B (16)
>>>> >> >          - NetworkThroughput(*): 373.13 KB/sec
>>>> >> >          - OverallThroughput: 63.53 KB/sec
>>>> >> >          - PeakMemoryUsage: 16.00 KB (16384)
>>>> >> >          - RowsReturned: 1 (1)
>>>> >> >          - SerializeBatchTime: 14.525us
>>>> >> >          - TransmitDataRPCTime: 43.122us
>>>> >> >          - UncompressedRowBatchSize: 16.00 B (16)
>>>> >> >       CodeGen:(Total: 41.687ms, non-child: 41.687ms, % non-child:
>>>> >> > 100.00%)
>>>> >> >          - CodegenTime: 1.194ms
>>>> >> >          - CompileTime: 7.360ms
>>>> >> >          - LoadTime: 0.000ns
>>>> >> >          - ModuleBitcodeSize: 1.86 MB (1950880)
>>>> >> >          - NumFunctions: 15 (15)
>>>> >> >          - NumInstructions: 214 (214)
>>>> >> >          - OptimizationTime: 13.529ms
>>>> >> >          - PeakMemoryUsage: 107.00 KB (109568)
>>>> >> >          - PrepareTime: 20.084ms
>>>> >> >       AGGREGATION_NODE (id=1):(Total: 283.825ms, non-child:
>>>> 1.027ms, %
>>>> >> > non-child: 0.36%)
>>>> >> >          - BuildTime: 1.259ms
>>>> >> >          - GetResultsTime: 0.000ns
>>>> >> >          - HTResizeTime: 0.000ns
>>>> >> >          - HashBuckets: 0 (0)
>>>> >> >          - LargestPartitionPercent: 0 (0)
>>>> >> >          - MaxPartitionLevel: 0 (0)
>>>> >> >          - NumRepartitions: 0 (0)
>>>> >> >          - PartitionsCreated: 0 (0)
>>>> >> >          - PeakMemoryUsage: 20.00 KB (20480)
>>>> >> >          - RowsRepartitioned: 0 (0)
>>>> >> >          - RowsReturned: 1 (1)
>>>> >> >          - RowsReturnedRate: 2.00 /sec
>>>> >> >          - SpilledPartitions: 0 (0)
>>>> >> >       HDFS_SCAN_NODE (id=0):(Total: 282.797ms, non-child:
>>>> 282.797ms, %
>>>> >> > non-child: 100.00%)
>>>> >> >          - AverageHdfsReadThreadConcurrency: 0.00
>>>> >> >          - AverageScannerThreadConcurrency: 14.40
>>>> >> >          - BytesRead: 44.52 MB (46686028)
>>>> >> >          - BytesReadDataNodeCache: 0
>>>> >> >          - BytesReadLocal: 44.52 MB (46686028)
>>>> >> >          - BytesReadRemoteUnexpected: 0
>>>> >> >          - BytesReadShortCircuit: 44.52 MB (46686028)
>>>> >> >          - CachedFileHandlesHitCount: 0 (0)
>>>> >> >          - CachedFileHandlesMissCount: 458 (458)
>>>> >> >          - CollectionItemsRead: 0 (0)
>>>> >> >          - DecompressionTime: 0.000ns
>>>> >> >          - MaxCompressedTextFileLength: 0
>>>> >> >          - NumColumns: 0 (0)
>>>> >> >          - NumDictFilteredRowGroups: 0 (0)
>>>> >> >          - NumDisksAccessed: 12 (12)
>>>> >> >          - NumRowGroups: 458 (458)
>>>> >> >          - NumScannerThreadsStarted: 24 (24)
>>>> >> >          - NumScannersWithNoReads: 0 (0)
>>>> >> >          - NumStatsFilteredRowGroups: 0 (0)
>>>> >> >          - PeakMemoryUsage: 2.50 MB (2625539)
>>>> >> >          - PerReadThreadRawHdfsThroughput: 1.29 GB/sec
>>>> >> >          - RemoteScanRanges: 0 (0)
>>>> >> >          - RowBatchQueueGetWaitTime: 269.787ms
>>>> >> >          - RowBatchQueuePutWaitTime: 0.000ns
>>>> >> >          - RowsRead: 0 (0)
>>>> >> >          - RowsReturned: 458 (458)
>>>> >> >          - RowsReturnedRate: 1.64 K/sec
>>>> >> >          - ScanRangesComplete: 458 (458)
>>>> >> >          - ScannerThreadsInvoluntaryContextSwitches: 99 (99)
>>>> >> >          - ScannerThreadsTotalWallClockTime: 6s532ms
>>>> >> >            - MaterializeTupleTime(*): 70.664us
>>>> >> >            - ScannerThreadsSysTime: 26.184ms
>>>> >> >            - ScannerThreadsUserTime: 650.089ms
>>>> >> >          - ScannerThreadsVoluntaryContextSwitches: 1.67K (1674)
>>>> >> >          - TotalRawHdfsReadTime(*): 34.416ms
>>>> >> >          - TotalReadThroughput: 36.86 MB/sec
>>>> >> >     Fragment F00:
>>>> >> >       Instance c843dae48ce9b289:6ad3ffc000000004
>>>> >> > (host=aopr-dhc004.lpdomain.com:22000):(Total: 377.384ms,
>>>> non-child:
>>>> >> > 1.619ms,
>>>> >> > % non-child: 0.43%)
>>>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> >> > 0:36/21.95 MB 11:33
>>>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/19.87 MB
>>>> 9:27 <https://maps.google.com/?q=MB+9:27&entry=gmail&source=g>/15.16 MB
>>>> 5:44 <https://maps.google.com/?q=MB+5:44&entry=gmail&source=g>/28.07 MB
>>>> 6:56 <https://maps.google.com/?q=MB+6:56&entry=gmail&source=g>/33.32 MB
>>>> >> > 1:38/22.24 MB 10:44
>>>> <https://maps.google.com/?q=MB+10:44&entry=gmail&source=g>/27.90 MB
>>>> 3:34 <https://maps.google.com/?q=MB+3:34&entry=gmail&source=g>/20.26 MB
>>>> 2:45 <https://maps.google.com/?q=MB+2:45&entry=gmail&source=g>/28.21 MB
>>>> 7:31 <https://maps.google.com/?q=MB+7:31&entry=gmail&source=g>/19.33 MB
>>>> >> > 4:42/26.73 MB 8:34
>>>> <https://maps.google.com/?q=MB+8:34&entry=gmail&source=g>/21.25 MB
>>>> >> >         MemoryUsage(500.000ms): 1.91 MB
>>>> >> >         ThreadUsage(500.000ms): 25
>>>> >> >          - AverageThreadTokens: 25.00
>>>> >> >          - BloomFilterBytes: 0
>>>> >> >          - PeakMemoryUsage: 2.41 MB (2531689)
>>>> >> >          - PeakReservation: 0
>>>> >> >          - PeakUsedReservation: 0
>>>> >> >          - PerHostPeakMemUsage: 2.41 MB (2531689)
>>>> >> >          - RowsProduced: 1 (1)
>>>> >> >          - TotalNetworkReceiveTime: 0.000ns
>>>> >> >          - TotalNetworkSendTime: 382.629us
>>>> >> >          - TotalStorageWaitTime: 5s790ms
>>>> >> >          - TotalThreadsInvoluntaryContextSwitches: 115 (115)
>>>> >> >          - TotalThreadsTotalWallClockTime: 8s122ms
>>>> >> >            - TotalThreadsSysTime: 33.984ms
>>>> >> >            - TotalThreadsUserTime: 702.879ms
>>>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.04K (2037)
>>>> >> >         Fragment Instance Lifecycle Timings:
>>>> >> >            - ExecTime: 604.005us
>>>> >> >              - ExecTreeExecTime: 8.338us
>>>> >> >            - OpenTime: 352.927ms
>>>> >> >              - ExecTreeOpenTime: 331.921ms
>>>> >> >            - PrepareTime: 23.783ms
>>>> >> >              - ExecTreePrepareTime: 2.559ms
>>>> >> >         DataStreamSender (dst_id=2):(Total: 331.818us, non-child:
>>>> >> > 331.818us,
>>>> >> > % non-child: 100.00%)
>>>> >> >            - BytesSent: 16.00 B (16)
>>>> >> >            - NetworkThroughput(*): 289.00 KB/sec
>>>> >> >            - OverallThroughput: 47.09 KB/sec
>>>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - SerializeBatchTime: 14.466us
>>>> >> >            - TransmitDataRPCTime: 54.066us
>>>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>>>> >> >         CodeGen:(Total: 40.950ms, non-child: 40.950ms, % non-child:
>>>> >> > 100.00%)
>>>> >> >            - CodegenTime: 1.122ms
>>>> >> >            - CompileTime: 7.318ms
>>>> >> >            - LoadTime: 0.000ns
>>>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>>>> >> >            - NumFunctions: 15 (15)
>>>> >> >            - NumInstructions: 214 (214)
>>>> >> >            - OptimizationTime: 13.106ms
>>>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>>>> >> >            - PrepareTime: 19.773ms
>>>> >> >         AGGREGATION_NODE (id=1):(Total: 334.482ms, non-child:
>>>> 766.527us,
>>>> >> > %
>>>> >> > non-child: 0.23%)
>>>> >> >           ExecOption: Codegen Enabled
>>>> >> >            - BuildTime: 1.192ms
>>>> >> >            - GetResultsTime: 0.000ns
>>>> >> >            - HTResizeTime: 0.000ns
>>>> >> >            - HashBuckets: 0 (0)
>>>> >> >            - LargestPartitionPercent: 0 (0)
>>>> >> >            - MaxPartitionLevel: 0 (0)
>>>> >> >            - NumRepartitions: 0 (0)
>>>> >> >            - PartitionsCreated: 0 (0)
>>>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>>>> >> >            - RowsRepartitioned: 0 (0)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - RowsReturnedRate: 2.00 /sec
>>>> >> >            - SpilledPartitions: 0 (0)
>>>> >> >         HDFS_SCAN_NODE (id=0):(Total: 333.715ms, non-child:
>>>> 333.715ms, %
>>>> >> > non-child: 100.00%)
>>>> >> >           Hdfs split stats (<volume id>:<# splits>/<split
>>>> lengths>):
>>>> >> > 0:36/21.95 MB 11:33
>>>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/19.87 MB
>>>> 9:27 <https://maps.google.com/?q=MB+9:27&entry=gmail&source=g>/15.16 MB
>>>> 5:44 <https://maps.google.com/?q=MB+5:44&entry=gmail&source=g>/28.07 MB
>>>> 6:56 <https://maps.google.com/?q=MB+6:56&entry=gmail&source=g>/33.32 MB
>>>> >> > 1:38/22.24 MB 10:44
>>>> <https://maps.google.com/?q=MB+10:44&entry=gmail&source=g>/27.90 MB
>>>> 3:34 <https://maps.google.com/?q=MB+3:34&entry=gmail&source=g>/20.26 MB
>>>> 2:45 <https://maps.google.com/?q=MB+2:45&entry=gmail&source=g>/28.21 MB
>>>> 7:31 <https://maps.google.com/?q=MB+7:31&entry=gmail&source=g>/19.33 MB
>>>> >> > 4:42/26.73 MB 8:34
>>>> <https://maps.google.com/?q=MB+8:34&entry=gmail&source=g>/21.25 MB
>>>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled:
>>>> 464 out
>>>> >> > of
>>>> >> > 464
>>>> >> >           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0%
>>>> 3:0%
>>>> >> > 4:0%
>>>> >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>> >> >           File Formats: PARQUET/Unknown(Skipped):464
>>>> >> >           BytesRead(500.000ms): 12.27 MB
>>>> >> >            - FooterProcessingTime: (Avg: 13.555ms ; Min: 5.433ms ;
>>>> Max:
>>>> >> > 62.957ms ; Number of samples: 464)
>>>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>>>> >> >            - AverageScannerThreadConcurrency: 24.00
>>>> >> >            - BytesRead: 45.01 MB (47193988)
>>>> >> >            - BytesReadDataNodeCache: 0
>>>> >> >            - BytesReadLocal: 45.01 MB (47193988)
>>>> >> >            - BytesReadRemoteUnexpected: 0
>>>> >> >            - BytesReadShortCircuit: 45.01 MB (47193988)
>>>> >> >            - CachedFileHandlesHitCount: 0 (0)
>>>> >> >            - CachedFileHandlesMissCount: 464 (464)
>>>> >> >            - CollectionItemsRead: 0 (0)
>>>> >> >            - DecompressionTime: 0.000ns
>>>> >> >            - MaxCompressedTextFileLength: 0
>>>> >> >            - NumColumns: 0 (0)
>>>> >> >            - NumDictFilteredRowGroups: 0 (0)
>>>> >> >            - NumDisksAccessed: 12 (12)
>>>> >> >            - NumRowGroups: 464 (464)
>>>> >> >            - NumScannerThreadsStarted: 24 (24)
>>>> >> >            - NumScannersWithNoReads: 0 (0)
>>>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>>>> >> >            - PeakMemoryUsage: 2.38 MB (2490376)
>>>> >> >            - PerReadThreadRawHdfsThroughput: 1016.44 MB/sec
>>>> >> >            - RemoteScanRanges: 0 (0)
>>>> >> >            - RowBatchQueueGetWaitTime: 320.860ms
>>>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>>>> >> >            - RowsRead: 0 (0)
>>>> >> >            - RowsReturned: 464 (464)
>>>> >> >            - RowsReturnedRate: 1.39 K/sec
>>>> >> >            - ScanRangesComplete: 464 (464)
>>>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 108 (108)
>>>> >> >            - ScannerThreadsTotalWallClockTime: 7s769ms
>>>> >> >              - MaterializeTupleTime(*): 75.514us
>>>> >> >              - ScannerThreadsSysTime: 29.984ms
>>>> >> >              - ScannerThreadsUserTime: 669.884ms
>>>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.60K (1597)
>>>> >> >            - TotalRawHdfsReadTime(*): 44.279ms
>>>> >> >            - TotalReadThroughput: 24.53 MB/sec
>>>> >> >       Instance c843dae48ce9b289:6ad3ffc000000002
>>>> >> > (host=aopr-dhc003.lpdomain.com:22000):(Total: 351.014ms,
>>>> non-child:
>>>> >> > 1.641ms,
>>>> >> > % non-child: 0.47%)
>>>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> >> > 0:35/20.89 MB 7:44
>>>> <https://maps.google.com/?q=MB+7:44&entry=gmail&source=g>/26.64 MB
>>>> 11:27 <https://maps.google.com/?q=MB+11:27&entry=gmail&source=g>/17.21 MB
>>>> 9:32 <https://maps.google.com/?q=MB+9:32&entry=gmail&source=g>/18.61 MB
>>>> 5:25 <https://maps.google.com/?q=MB+5:25&entry=gmail&source=g>/14.56 MB
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+2:38&entry=gmail&source=g>
>>>> >> > 2:38
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+2:38&entry=gmail&source=g>/24.32
>>>> MB 6:31 <https://maps.google.com/?q=MB+6:31&entry=gmail&source=g>/19.85
>>>> MB 8:50 <https://maps.google.com/?q=MB+8:50&entry=gmail&source=g>/30.76
>>>> MB 10:41 <https://maps.google.com/?q=MB+10:41&entry=gmail&source=g>/25.92
>>>> MB 3:31 <https://maps.google.com/?q=MB+3:31&entry=gmail&source=g>/19.78
>>>> MB
>>>> >> > 4:60/37.59 MB 1:44/27.19 MB
>>>> >> >         MemoryUsage(500.000ms): 1.16 MB
>>>> >> >         ThreadUsage(500.000ms): 25
>>>> >> >          - AverageThreadTokens: 25.00
>>>> >> >          - BloomFilterBytes: 0
>>>> >> >          - PeakMemoryUsage: 2.54 MB (2662753)
>>>> >> >          - PeakReservation: 0
>>>> >> >          - PeakUsedReservation: 0
>>>> >> >          - PerHostPeakMemUsage: 2.54 MB (2662753)
>>>> >> >          - RowsProduced: 1 (1)
>>>> >> >          - TotalNetworkReceiveTime: 0.000ns
>>>> >> >          - TotalNetworkSendTime: 342.746us
>>>> >> >          - TotalStorageWaitTime: 4s796ms
>>>> >> >          - TotalThreadsInvoluntaryContextSwitches: 93 (93)
>>>> >> >          - TotalThreadsTotalWallClockTime: 7s286ms
>>>> >> >            - TotalThreadsSysTime: 27.988ms
>>>> >> >            - TotalThreadsUserTime: 671.887ms
>>>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.08K (2079)
>>>> >> >         Fragment Instance Lifecycle Timings:
>>>> >> >            - ExecTime: 524.006us
>>>> >> >              - ExecTreeExecTime: 7.920us
>>>> >> >            - OpenTime: 327.802ms
>>>> >> >              - ExecTreeOpenTime: 307.154ms
>>>> >> >            - PrepareTime: 22.617ms
>>>> >> >              - ExecTreePrepareTime: 2.650ms
>>>> >> >         DataStreamSender (dst_id=2):(Total: 348.990us, non-child:
>>>> >> > 348.990us,
>>>> >> > % non-child: 100.00%)
>>>> >> >            - BytesSent: 16.00 B (16)
>>>> >> >            - NetworkThroughput(*): 354.47 KB/sec
>>>> >> >            - OverallThroughput: 44.77 KB/sec
>>>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - SerializeBatchTime: 15.602us
>>>> >> >            - TransmitDataRPCTime: 44.080us
>>>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>>>> >> >         CodeGen:(Total: 39.215ms, non-child: 39.215ms, % non-child:
>>>> >> > 100.00%)
>>>> >> >            - CodegenTime: 1.183ms
>>>> >> >            - CompileTime: 7.168ms
>>>> >> >            - LoadTime: 0.000ns
>>>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>>>> >> >            - NumFunctions: 15 (15)
>>>> >> >            - NumInstructions: 214 (214)
>>>> >> >            - OptimizationTime: 12.929ms
>>>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>>>> >> >            - PrepareTime: 18.453ms
>>>> >> >         AGGREGATION_NODE (id=1):(Total: 309.807ms, non-child:
>>>> 948.677us,
>>>> >> > %
>>>> >> > non-child: 0.31%)
>>>> >> >           ExecOption: Codegen Enabled
>>>> >> >            - BuildTime: 1.552ms
>>>> >> >            - GetResultsTime: 0.000ns
>>>> >> >            - HTResizeTime: 0.000ns
>>>> >> >            - HashBuckets: 0 (0)
>>>> >> >            - LargestPartitionPercent: 0 (0)
>>>> >> >            - MaxPartitionLevel: 0 (0)
>>>> >> >            - NumRepartitions: 0 (0)
>>>> >> >            - PartitionsCreated: 0 (0)
>>>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>>>> >> >            - RowsRepartitioned: 0 (0)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - RowsReturnedRate: 3.00 /sec
>>>> >> >            - SpilledPartitions: 0 (0)
>>>> >> >         HDFS_SCAN_NODE (id=0):(Total: 308.859ms, non-child:
>>>> 308.859ms, %
>>>> >> > non-child: 100.00%)
>>>> >> >           Hdfs split stats (<volume id>:<# splits>/<split
>>>> lengths>):
>>>> >> > 0:35/20.89 MB 7:44
>>>> <https://maps.google.com/?q=MB+7:44&entry=gmail&source=g>/26.64 MB
>>>> 11:27 <https://maps.google.com/?q=MB+11:27&entry=gmail&source=g>/17.21 MB
>>>> 9:32 <https://maps.google.com/?q=MB+9:32&entry=gmail&source=g>/18.61 MB
>>>> 5:25 <https://maps.google.com/?q=MB+5:25&entry=gmail&source=g>/14.56 MB
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+2:38&entry=gmail&source=g>
>>>> >> > 2:38
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+2:38&entry=gmail&source=g>/24.32
>>>> MB 6:31 <https://maps.google.com/?q=MB+6:31&entry=gmail&source=g>/19.85
>>>> MB 8:50 <https://maps.google.com/?q=MB+8:50&entry=gmail&source=g>/30.76
>>>> MB 10:41 <https://maps.google.com/?q=MB+10:41&entry=gmail&source=g>/25.92
>>>> MB 3:31 <https://maps.google.com/?q=MB+3:31&entry=gmail&source=g>/19.78
>>>> MB
>>>> >> > 4:60/37.59 MB 1:44/27.19 MB
>>>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled:
>>>> 458 out
>>>> >> > of
>>>> >> > 458
>>>> >> >           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0%
>>>> 3:0%
>>>> >> > 4:0%
>>>> >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>> >> >           File Formats: PARQUET/Unknown(Skipped):458
>>>> >> >           BytesRead(500.000ms): 36.74 MB
>>>> >> >            - FooterProcessingTime: (Avg: 11.502ms ; Min: 3.668ms ;
>>>> Max:
>>>> >> > 23.564ms ; Number of samples: 458)
>>>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>>>> >> >            - AverageScannerThreadConcurrency: 24.00
>>>> >> >            - BytesRead: 44.51 MB (46668432)
>>>> >> >            - BytesReadDataNodeCache: 0
>>>> >> >            - BytesReadLocal: 44.51 MB (46668432)
>>>> >> >            - BytesReadRemoteUnexpected: 0
>>>> >> >            - BytesReadShortCircuit: 44.51 MB (46668432)
>>>> >> >            - CachedFileHandlesHitCount: 0 (0)
>>>> >> >            - CachedFileHandlesMissCount: 458 (458)
>>>> >> >            - CollectionItemsRead: 0 (0)
>>>> >> >            - DecompressionTime: 0.000ns
>>>> >> >            - MaxCompressedTextFileLength: 0
>>>> >> >            - NumColumns: 0 (0)
>>>> >> >            - NumDictFilteredRowGroups: 0 (0)
>>>> >> >            - NumDisksAccessed: 12 (12)
>>>> >> >            - NumRowGroups: 458 (458)
>>>> >> >            - NumScannerThreadsStarted: 24 (24)
>>>> >> >            - NumScannersWithNoReads: 0 (0)
>>>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>>>> >> >            - PeakMemoryUsage: 2.50 MB (2621440)
>>>> >> >            - PerReadThreadRawHdfsThroughput: 1.26 GB/sec
>>>> >> >            - RemoteScanRanges: 0 (0)
>>>> >> >            - RowBatchQueueGetWaitTime: 295.595ms
>>>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>>>> >> >            - RowsRead: 0 (0)
>>>> >> >            - RowsReturned: 458 (458)
>>>> >> >            - RowsReturnedRate: 1.48 K/sec
>>>> >> >            - ScanRangesComplete: 458 (458)
>>>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 91 (91)
>>>> >> >            - ScannerThreadsTotalWallClockTime: 6s958ms
>>>> >> >              - MaterializeTupleTime(*): 71.630us
>>>> >> >              - ScannerThreadsSysTime: 16.989ms
>>>> >> >              - ScannerThreadsUserTime: 645.891ms
>>>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.63K (1630)
>>>> >> >            - TotalRawHdfsReadTime(*): 34.377ms
>>>> >> >            - TotalReadThroughput: 73.49 MB/sec
>>>> >> >       Instance c843dae48ce9b289:6ad3ffc000000001
>>>> >> > (host=aopr-dhc005.lpdomain.com:22000):(Total: 308.195ms,
>>>> non-child:
>>>> >> > 1.784ms,
>>>> >> > % non-child: 0.58%)
>>>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> >> > 2:46/28.97 MB 8:39
>>>> <https://maps.google.com/?q=MB+8:39&entry=gmail&source=g>/24.34 MB
>>>> 10:33 <https://maps.google.com/?q=MB+10:33&entry=gmail&source=g>/21.03 MB
>>>> 3:43 <https://maps.google.com/?q=MB+3:43&entry=gmail&source=g>/26.50 MB
>>>> 5:32 <https://maps.google.com/?q=MB+5:32&entry=gmail&source=g>/20.27 MB
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>
>>>> >> > 9:32
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>/19.86
>>>> MB 6:25 <https://maps.google.com/?q=MB+6:25&entry=gmail&source=g>/16.01
>>>> MB 7:41 <https://maps.google.com/?q=MB+7:41&entry=gmail&source=g>/24.77
>>>> MB 1:42/25.01 MB 0:55/32.51 MB
>>>> >> > 4:40/24.29 MB 11:33
>>>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/20.77 MB
>>>> >> >          - AverageThreadTokens: 0.00
>>>> >> >          - BloomFilterBytes: 0
>>>> >> >          - PeakMemoryUsage: 2.56 MB (2683241)
>>>> >> >          - PeakReservation: 0
>>>> >> >          - PeakUsedReservation: 0
>>>> >> >          - PerHostPeakMemUsage: 2.56 MB (2683241)
>>>> >> >          - RowsProduced: 1 (1)
>>>> >> >          - TotalNetworkReceiveTime: 0.000ns
>>>> >> >          - TotalNetworkSendTime: 292.765us
>>>> >> >          - TotalStorageWaitTime: 4s482ms
>>>> >> >          - TotalThreadsInvoluntaryContextSwitches: 113 (113)
>>>> >> >          - TotalThreadsTotalWallClockTime: 6s310ms
>>>> >> >            - TotalThreadsSysTime: 35.983ms
>>>> >> >            - TotalThreadsUserTime: 682.884ms
>>>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.14K (2143)
>>>> >> >         Fragment Instance Lifecycle Timings:
>>>> >> >            - ExecTime: 532.572us
>>>> >> >              - ExecTreeExecTime: 8.377us
>>>> >> >            - OpenTime: 280.241ms
>>>> >> >              - ExecTreeOpenTime: 258.652ms
>>>> >> >            - PrepareTime: 27.342ms
>>>> >> >              - ExecTreePrepareTime: 2.863ms
>>>> >> >         DataStreamSender (dst_id=2):(Total: 309.794us, non-child:
>>>> >> > 309.794us,
>>>> >> > % non-child: 100.00%)
>>>> >> >            - BytesSent: 16.00 B (16)
>>>> >> >            - NetworkThroughput(*): 356.91 KB/sec
>>>> >> >            - OverallThroughput: 50.44 KB/sec
>>>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - SerializeBatchTime: 14.262us
>>>> >> >            - TransmitDataRPCTime: 43.778us
>>>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>>>> >> >         CodeGen:(Total: 44.584ms, non-child: 44.584ms, % non-child:
>>>> >> > 100.00%)
>>>> >> >            - CodegenTime: 1.382ms
>>>> >> >            - CompileTime: 7.346ms
>>>> >> >            - LoadTime: 0.000ns
>>>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>>>> >> >            - NumFunctions: 15 (15)
>>>> >> >            - NumInstructions: 214 (214)
>>>> >> >            - OptimizationTime: 13.462ms
>>>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>>>> >> >            - PrepareTime: 23.006ms
>>>> >> >         AGGREGATION_NODE (id=1):(Total: 261.516ms, non-child:
>>>> 836.656us,
>>>> >> > %
>>>> >> > non-child: 0.32%)
>>>> >> >           ExecOption: Codegen Enabled
>>>> >> >            - BuildTime: 1.280ms
>>>> >> >            - GetResultsTime: 0.000ns
>>>> >> >            - HTResizeTime: 0.000ns
>>>> >> >            - HashBuckets: 0 (0)
>>>> >> >            - LargestPartitionPercent: 0 (0)
>>>> >> >            - MaxPartitionLevel: 0 (0)
>>>> >> >            - NumRepartitions: 0 (0)
>>>> >> >            - PartitionsCreated: 0 (0)
>>>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>>>> >> >            - RowsRepartitioned: 0 (0)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - RowsReturnedRate: 3.00 /sec
>>>> >> >            - SpilledPartitions: 0 (0)
>>>> >> >         HDFS_SCAN_NODE (id=0):(Total: 260.680ms, non-child:
>>>> 260.680ms, %
>>>> >> > non-child: 100.00%)
>>>> >> >           Hdfs split stats (<volume id>:<# splits>/<split
>>>> lengths>):
>>>> >> > 2:46/28.97 MB 8:39
>>>> <https://maps.google.com/?q=MB+8:39&entry=gmail&source=g>/24.34 MB
>>>> 10:33 <https://maps.google.com/?q=MB+10:33&entry=gmail&source=g>/21.03 MB
>>>> 3:43 <https://maps.google.com/?q=MB+3:43&entry=gmail&source=g>/26.50 MB
>>>> 5:32 <https://maps.google.com/?q=MB+5:32&entry=gmail&source=g>/20.27 MB
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>
>>>> >> > 9:32
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>/19.86
>>>> MB 6:25 <https://maps.google.com/?q=MB+6:25&entry=gmail&source=g>/16.01
>>>> MB 7:41 <https://maps.google.com/?q=MB+7:41&entry=gmail&source=g>/24.77
>>>> MB 1:42/25.01 MB 0:55/32.51 MB
>>>> >> > 4:40/24.29 MB 11:33
>>>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/20.77 MB
>>>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled:
>>>> 461 out
>>>> >> > of
>>>> >> > 461
>>>> >> >           Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0%
>>>> 4:0%
>>>> >> > 5:0%
>>>> >> > 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>> >> >           File Formats: PARQUET/Unknown(Skipped):461
>>>> >> >            - FooterProcessingTime: (Avg: 10.793ms ; Min: 3.087ms ;
>>>> Max:
>>>> >> > 20.253ms ; Number of samples: 461)
>>>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>>>> >> >            - AverageScannerThreadConcurrency: 0.00
>>>> >> >            - BytesRead: 44.73 MB (46906407)
>>>> >> >            - BytesReadDataNodeCache: 0
>>>> >> >            - BytesReadLocal: 44.73 MB (46906407)
>>>> >> >            - BytesReadRemoteUnexpected: 0
>>>> >> >            - BytesReadShortCircuit: 44.73 MB (46906407)
>>>> >> >            - CachedFileHandlesHitCount: 0 (0)
>>>> >> >            - CachedFileHandlesMissCount: 461 (461)
>>>> >> >            - CollectionItemsRead: 0 (0)
>>>> >> >            - DecompressionTime: 0.000ns
>>>> >> >            - MaxCompressedTextFileLength: 0
>>>> >> >            - NumColumns: 0 (0)
>>>> >> >            - NumDictFilteredRowGroups: 0 (0)
>>>> >> >            - NumDisksAccessed: 12 (12)
>>>> >> >            - NumRowGroups: 461 (461)
>>>> >> >            - NumScannerThreadsStarted: 24 (24)
>>>> >> >            - NumScannersWithNoReads: 0 (0)
>>>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>>>> >> >            - PeakMemoryUsage: 2.52 MB (2641928)
>>>> >> >            - PerReadThreadRawHdfsThroughput: 1.31 GB/sec
>>>> >> >            - RemoteScanRanges: 0 (0)
>>>> >> >            - RowBatchQueueGetWaitTime: 246.691ms
>>>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>>>> >> >            - RowsRead: 0 (0)
>>>> >> >            - RowsReturned: 461 (461)
>>>> >> >            - RowsReturnedRate: 1.77 K/sec
>>>> >> >            - ScanRangesComplete: 461 (461)
>>>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 109 (109)
>>>> >> >            - ScannerThreadsTotalWallClockTime: 6s030ms
>>>> >> >              - MaterializeTupleTime(*): 68.735us
>>>> >> >              - ScannerThreadsSysTime: 33.983ms
>>>> >> >              - ScannerThreadsUserTime: 647.889ms
>>>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.70K (1696)
>>>> >> >            - TotalRawHdfsReadTime(*): 33.405ms
>>>> >> >            - TotalReadThroughput: 0.00 /sec
>>>> >> >       Instance c843dae48ce9b289:6ad3ffc000000003
>>>> >> > (host=aopr-dhc002.lpdomain.com:22000):(Total: 302.116ms,
>>>> non-child:
>>>> >> > 1.604ms,
>>>> >> > % non-child: 0.53%)
>>>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> >> > 0:27/16.21 MB 11:33
>>>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/21.23 MB
>>>> 6:36 <https://maps.google.com/?q=MB+6:36&entry=gmail&source=g>/23.15 MB
>>>> 10:41 <https://maps.google.com/?q=MB+10:41&entry=gmail&source=g>/25.09
>>>> MB 1:41/24.13 MB
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>
>>>> >> > 9:32
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>/18.79
>>>> MB 5:41 <https://maps.google.com/?q=MB+5:41&entry=gmail&source=g>/25.31
>>>> MB 2:42 <https://maps.google.com/?q=MB+2:42&entry=gmail&source=g>/26.49
>>>> MB 7:36 <https://maps.google.com/?q=MB+7:36&entry=gmail&source=g>/22.87
>>>> MB 3:51 <https://maps.google.com/?q=MB+3:51&entry=gmail&source=g>/31.00
>>>> MB
>>>> >> > 4:41/25.99 MB 8:38
>>>> <https://maps.google.com/?q=MB+8:38&entry=gmail&source=g>/23.79 MB
>>>> >> >         MemoryUsage(500.000ms): 700.35 KB
>>>> >> >         ThreadUsage(500.000ms): 25
>>>> >> >          - AverageThreadTokens: 25.00
>>>> >> >          - BloomFilterBytes: 0
>>>> >> >          - PeakMemoryUsage: 2.54 MB (2662753)
>>>> >> >          - PeakReservation: 0
>>>> >> >          - PeakUsedReservation: 0
>>>> >> >          - PerHostPeakMemUsage: 2.54 MB (2662753)
>>>> >> >          - RowsProduced: 1 (1)
>>>> >> >          - TotalNetworkReceiveTime: 0.000ns
>>>> >> >          - TotalNetworkSendTime: 300.675us
>>>> >> >          - TotalStorageWaitTime: 4s661ms
>>>> >> >          - TotalThreadsInvoluntaryContextSwitches: 90 (90)
>>>> >> >          - TotalThreadsTotalWallClockTime: 6s256ms
>>>> >> >            - TotalThreadsSysTime: 30.983ms
>>>> >> >            - TotalThreadsUserTime: 709.880ms
>>>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.17K (2175)
>>>> >> >         Fragment Instance Lifecycle Timings:
>>>> >> >            - ExecTime: 492.627us
>>>> >> >              - ExecTreeExecTime: 8.191us
>>>> >> >            - OpenTime: 277.922ms
>>>> >> >              - ExecTreeOpenTime: 255.395ms
>>>> >> >            - PrepareTime: 23.647ms
>>>> >> >              - ExecTreePrepareTime: 2.243ms
>>>> >> >         DataStreamSender (dst_id=2):(Total: 194.517us, non-child:
>>>> >> > 194.517us,
>>>> >> > % non-child: 100.00%)
>>>> >> >            - BytesSent: 16.00 B (16)
>>>> >> >            - NetworkThroughput(*): 371.86 KB/sec
>>>> >> >            - OverallThroughput: 80.33 KB/sec
>>>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - SerializeBatchTime: 14.035us
>>>> >> >            - TransmitDataRPCTime: 42.019us
>>>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>>>> >> >         CodeGen:(Total: 42.676ms, non-child: 42.676ms, % non-child:
>>>> >> > 100.00%)
>>>> >> >            - CodegenTime: 1.236ms
>>>> >> >            - CompileTime: 7.717ms
>>>> >> >            - LoadTime: 0.000ns
>>>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>>>> >> >            - NumFunctions: 15 (15)
>>>> >> >            - NumInstructions: 214 (214)
>>>> >> >            - OptimizationTime: 14.130ms
>>>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>>>> >> >            - PrepareTime: 20.109ms
>>>> >> >         AGGREGATION_NODE (id=1):(Total: 257.640ms, non-child:
>>>> 1.541ms, %
>>>> >> > non-child: 0.60%)
>>>> >> >           ExecOption: Codegen Enabled
>>>> >> >            - BuildTime: 1.252ms
>>>> >> >            - GetResultsTime: 0.000ns
>>>> >> >            - HTResizeTime: 0.000ns
>>>> >> >            - HashBuckets: 0 (0)
>>>> >> >            - LargestPartitionPercent: 0 (0)
>>>> >> >            - MaxPartitionLevel: 0 (0)
>>>> >> >            - NumRepartitions: 0 (0)
>>>> >> >            - PartitionsCreated: 0 (0)
>>>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>>>> >> >            - RowsRepartitioned: 0 (0)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - RowsReturnedRate: 3.00 /sec
>>>> >> >            - SpilledPartitions: 0 (0)
>>>> >> >         HDFS_SCAN_NODE (id=0):(Total: 256.099ms, non-child:
>>>> 256.099ms, %
>>>> >> > non-child: 100.00%)
>>>> >> >           Hdfs split stats (<volume id>:<# splits>/<split
>>>> lengths>):
>>>> >> > 0:27/16.21 MB 11:33
>>>> <https://maps.google.com/?q=MB+11:33&entry=gmail&source=g>/21.23 MB
>>>> 6:36 <https://maps.google.com/?q=MB+6:36&entry=gmail&source=g>/23.15 MB
>>>> 10:41 <https://maps.google.com/?q=MB+10:41&entry=gmail&source=g>/25.09
>>>> MB 1:41/24.13 MB
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>
>>>> >> > 9:32
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+9:32&entry=gmail&source=g>/18.79
>>>> MB 5:41 <https://maps.google.com/?q=MB+5:41&entry=gmail&source=g>/25.31
>>>> MB 2:42 <https://maps.google.com/?q=MB+2:42&entry=gmail&source=g>/26.49
>>>> MB 7:36 <https://maps.google.com/?q=MB+7:36&entry=gmail&source=g>/22.87
>>>> MB 3:51 <https://maps.google.com/?q=MB+3:51&entry=gmail&source=g>/31.00
>>>> MB
>>>> >> > 4:41/25.99 MB 8:38
>>>> <https://maps.google.com/?q=MB+8:38&entry=gmail&source=g>/23.79 MB
>>>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled:
>>>> 459 out
>>>> >> > of
>>>> >> > 459
>>>> >> >           Hdfs Read Thread Concurrency Bucket: 0:100% 1:0% 2:0%
>>>> 3:0%
>>>> >> > 4:0%
>>>> >> > 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>> >> >           File Formats: PARQUET/Unknown(Skipped):459
>>>> >> >           BytesRead(500.000ms): 43.15 MB
>>>> >> >            - FooterProcessingTime: (Avg: 11.257ms ; Min: 3.989ms ;
>>>> Max:
>>>> >> > 23.004ms ; Number of samples: 459)
>>>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>>>> >> >            - AverageScannerThreadConcurrency: 24.00
>>>> >> >            - BytesRead: 44.52 MB (46678031)
>>>> >> >            - BytesReadDataNodeCache: 0
>>>> >> >            - BytesReadLocal: 44.52 MB (46678031)
>>>> >> >            - BytesReadRemoteUnexpected: 0
>>>> >> >            - BytesReadShortCircuit: 44.52 MB (46678031)
>>>> >> >            - CachedFileHandlesHitCount: 0 (0)
>>>> >> >            - CachedFileHandlesMissCount: 459 (459)
>>>> >> >            - CollectionItemsRead: 0 (0)
>>>> >> >            - DecompressionTime: 0.000ns
>>>> >> >            - MaxCompressedTextFileLength: 0
>>>> >> >            - NumColumns: 0 (0)
>>>> >> >            - NumDictFilteredRowGroups: 0 (0)
>>>> >> >            - NumDisksAccessed: 12 (12)
>>>> >> >            - NumRowGroups: 459 (459)
>>>> >> >            - NumScannerThreadsStarted: 24 (24)
>>>> >> >            - NumScannersWithNoReads: 0 (0)
>>>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>>>> >> >            - PeakMemoryUsage: 2.50 MB (2621440)
>>>> >> >            - PerReadThreadRawHdfsThroughput: 1.33 GB/sec
>>>> >> >            - RemoteScanRanges: 0 (0)
>>>> >> >            - RowBatchQueueGetWaitTime: 243.426ms
>>>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>>>> >> >            - RowsRead: 0 (0)
>>>> >> >            - RowsReturned: 459 (459)
>>>> >> >            - RowsReturnedRate: 1.79 K/sec
>>>> >> >            - ScanRangesComplete: 459 (459)
>>>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 85 (85)
>>>> >> >            - ScannerThreadsTotalWallClockTime: 5s978ms
>>>> >> >              - MaterializeTupleTime(*): 67.997us
>>>> >> >              - ScannerThreadsSysTime: 22.984ms
>>>> >> >              - ScannerThreadsUserTime: 678.884ms
>>>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.72K (1724)
>>>> >> >            - TotalRawHdfsReadTime(*): 32.626ms
>>>> >> >            - TotalReadThroughput: 86.30 MB/sec
>>>> >> >       Instance c843dae48ce9b289:6ad3ffc000000005
>>>> >> > (host=aopr-dhc001.lpdomain.com:22000):(Total: 298.093ms,
>>>> non-child:
>>>> >> > 1.242ms,
>>>> >> > % non-child: 0.42%)
>>>> >> >         Hdfs split stats (<volume id>:<# splits>/<split lengths>):
>>>> >> > 9:55/35.38 MB 0:37/24.23 MB 5:37
>>>> <https://maps.google.com/?q=MB+5:37&entry=gmail&source=g>/23.31 MB
>>>> 11:22 <https://maps.google.com/?q=MB+11:22&entry=gmail&source=g>/13.79 MB
>>>> 8:30 <https://maps.google.com/?q=MB+8:30&entry=gmail&source=g>/18.53 MB
>>>> >> > 4:33/20.45 MB 1:38/24.06 MB 10:29
>>>> <https://maps.google.com/?q=MB+10:29&entry=gmail&source=g>/18.40 MB
>>>> 3:39 <https://maps.google.com/?q=MB+3:39&entry=gmail&source=g>/24.35 MB
>>>> 2:41 <https://maps.google.com/?q=MB+2:41&entry=gmail&source=g>/25.45 MB
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+6:42&entry=gmail&source=g>
>>>> >> > 6:42
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+6:42&entry=gmail&source=g>/27.03
>>>> MB 7:47 <https://maps.google.com/?q=MB+7:47&entry=gmail&source=g>/29.22
>>>> MB
>>>> >> >          - AverageThreadTokens: 0.00
>>>> >> >          - BloomFilterBytes: 0
>>>> >> >          - PeakMemoryUsage: 2.66 MB (2793825)
>>>> >> >          - PeakReservation: 0
>>>> >> >          - PeakUsedReservation: 0
>>>> >> >          - PerHostPeakMemUsage: 2.68 MB (2806113)
>>>> >> >          - RowsProduced: 1 (1)
>>>> >> >          - TotalNetworkReceiveTime: 0.000ns
>>>> >> >          - TotalNetworkSendTime: 198.894us
>>>> >> >          - TotalStorageWaitTime: 4s374ms
>>>> >> >          - TotalThreadsInvoluntaryContextSwitches: 110 (110)
>>>> >> >          - TotalThreadsTotalWallClockTime: 6s201ms
>>>> >> >            - TotalThreadsSysTime: 29.983ms
>>>> >> >            - TotalThreadsUserTime: 642.892ms
>>>> >> >          - TotalThreadsVoluntaryContextSwitches: 2.17K (2174)
>>>> >> >         Fragment Instance Lifecycle Timings:
>>>> >> >            - ExecTime: 350.873us
>>>> >> >              - ExecTreeExecTime: 8.173us
>>>> >> >            - OpenTime: 275.347ms
>>>> >> >              - ExecTreeOpenTime: 253.425ms
>>>> >> >            - PrepareTime: 22.341ms
>>>> >> >              - ExecTreePrepareTime: 2.251ms
>>>> >> >         DataStreamSender (dst_id=2):(Total: 164.394us, non-child:
>>>> >> > 164.394us,
>>>> >> > % non-child: 100.00%)
>>>> >> >            - BytesSent: 16.00 B (16)
>>>> >> >            - NetworkThroughput(*): 493.42 KB/sec
>>>> >> >            - OverallThroughput: 95.05 KB/sec
>>>> >> >            - PeakMemoryUsage: 16.00 KB (16384)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - SerializeBatchTime: 14.262us
>>>> >> >            - TransmitDataRPCTime: 31.667us
>>>> >> >            - UncompressedRowBatchSize: 16.00 B (16)
>>>> >> >         CodeGen:(Total: 41.008ms, non-child: 41.008ms, % non-child:
>>>> >> > 100.00%)
>>>> >> >            - CodegenTime: 1.049ms
>>>> >> >            - CompileTime: 7.248ms
>>>> >> >            - LoadTime: 0.000ns
>>>> >> >            - ModuleBitcodeSize: 1.86 MB (1950880)
>>>> >> >            - NumFunctions: 15 (15)
>>>> >> >            - NumInstructions: 214 (214)
>>>> >> >            - OptimizationTime: 14.018ms
>>>> >> >            - PeakMemoryUsage: 107.00 KB (109568)
>>>> >> >            - PrepareTime: 19.080ms
>>>> >> >         AGGREGATION_NODE (id=1):(Total: 255.678ms, non-child:
>>>> 1.046ms, %
>>>> >> > non-child: 0.41%)
>>>> >> >           ExecOption: Codegen Enabled
>>>> >> >            - BuildTime: 1.015ms
>>>> >> >            - GetResultsTime: 0.000ns
>>>> >> >            - HTResizeTime: 0.000ns
>>>> >> >            - HashBuckets: 0 (0)
>>>> >> >            - LargestPartitionPercent: 0 (0)
>>>> >> >            - MaxPartitionLevel: 0 (0)
>>>> >> >            - NumRepartitions: 0 (0)
>>>> >> >            - PartitionsCreated: 0 (0)
>>>> >> >            - PeakMemoryUsage: 20.00 KB (20480)
>>>> >> >            - RowsRepartitioned: 0 (0)
>>>> >> >            - RowsReturned: 1 (1)
>>>> >> >            - RowsReturnedRate: 3.00 /sec
>>>> >> >            - SpilledPartitions: 0 (0)
>>>> >> >         HDFS_SCAN_NODE (id=0):(Total: 254.631ms, non-child:
>>>> 254.631ms, %
>>>> >> > non-child: 100.00%)
>>>> >> >           Hdfs split stats (<volume id>:<# splits>/<split
>>>> lengths>):
>>>> >> > 9:55/35.38 MB 0:37/24.23 MB 5:37
>>>> <https://maps.google.com/?q=MB+5:37&entry=gmail&source=g>/23.31 MB
>>>> 11:22 <https://maps.google.com/?q=MB+11:22&entry=gmail&source=g>/13.79 MB
>>>> 8:30 <https://maps.google.com/?q=MB+8:30&entry=gmail&source=g>/18.53 MB
>>>> >> > 4:33/20.45 MB 1:38/24.06 MB 10:29
>>>> <https://maps.google.com/?q=MB+10:29&entry=gmail&source=g>/18.40 MB
>>>> 3:39 <https://maps.google.com/?q=MB+3:39&entry=gmail&source=g>/24.35 MB
>>>> 2:41 <https://maps.google.com/?q=MB+2:41&entry=gmail&source=g>/25.45 MB
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+6:42&entry=gmail&source=g>
>>>> >> > 6:42
>>>> <https://maps.google.com/?q=MB+%0D+%3E%3E+%3E+6:42&entry=gmail&source=g>/27.03
>>>> MB 7:47 <https://maps.google.com/?q=MB+7:47&entry=gmail&source=g>/29.22
>>>> MB
>>>> >> >           ExecOption: PARQUET Codegen Enabled, Codegen enabled:
>>>> 450 out
>>>> >> > of
>>>> >> > 450
>>>> >> >           Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0%
>>>> 4:0%
>>>> >> > 5:0%
>>>> >> > 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0% 15:0% 16:0%
>>>> >> >           File Formats: PARQUET/Unknown(Skipped):450
>>>> >> >            - FooterProcessingTime: (Avg: 10.727ms ; Min: 3.813ms ;
>>>> Max:
>>>> >> > 23.755ms ; Number of samples: 450)
>>>> >> >            - AverageHdfsReadThreadConcurrency: 0.00
>>>> >> >            - AverageScannerThreadConcurrency: 0.00
>>>> >> >            - BytesRead: 43.85 MB (45983284)
>>>> >> >            - BytesReadDataNodeCache: 0
>>>> >> >            - BytesReadLocal: 43.85 MB (45983284)
>>>> >> >            - BytesReadRemoteUnexpected: 0
>>>> >> >            - BytesReadShortCircuit: 43.85 MB (45983284)
>>>> >> >            - CachedFileHandlesHitCount: 0 (0)
>>>> >> >            - CachedFileHandlesMissCount: 450 (450)
>>>> >> >            - CollectionItemsRead: 0 (0)
>>>> >> >            - DecompressionTime: 0.000ns
>>>> >> >            - MaxCompressedTextFileLength: 0
>>>> >> >            - NumColumns: 0 (0)
>>>> >> >            - NumDictFilteredRowGroups: 0 (0)
>>>> >> >            - NumDisksAccessed: 12 (12)
>>>> >> >            - NumRowGroups: 450 (450)
>>>> >> >            - NumScannerThreadsStarted: 24 (24)
>>>> >> >            - NumScannersWithNoReads: 0 (0)
>>>> >> >            - NumStatsFilteredRowGroups: 0 (0)
>>>> >> >            - PeakMemoryUsage: 2.62 MB (2752512)
>>>> >> >            - PerReadThreadRawHdfsThroughput: 1.56 GB/sec
>>>> >> >            - RemoteScanRanges: 0 (0)
>>>> >> >            - RowBatchQueueGetWaitTime: 242.362ms
>>>> >> >            - RowBatchQueuePutWaitTime: 0.000ns
>>>> >> >            - RowsRead: 0 (0)
>>>> >> >            - RowsReturned: 450 (450)
>>>> >> >            - RowsReturnedRate: 1.77 K/sec
>>>> >> >            - ScanRangesComplete: 450 (450)
>>>> >> >            - ScannerThreadsInvoluntaryContextSwitches: 106 (106)
>>>> >> >            - ScannerThreadsTotalWallClockTime: 5s926ms
>>>> >> >              - MaterializeTupleTime(*): 69.447us
>>>> >> >              - ScannerThreadsSysTime: 26.984ms
>>>> >> >              - ScannerThreadsUserTime: 607.897ms
>>>> >> >            - ScannerThreadsVoluntaryContextSwitches: 1.72K (1723)
>>>> >> >            - TotalRawHdfsReadTime(*): 27.393ms
>>>> >> >            - TotalReadThroughput: 0.00 /sec
>>>> >> > ===================================================
>>>> >> >
>>>> >> > Coordinator log:
>>>> >> >
>>>> >> >  01: query (string) = "select count(*) from
>>>> dv_entersiteevent_prq_remote
>>>> >> > where year=2017 and month=10",
>>>> >> >   03: configuration (list) = list<string>[1] {
>>>> >> >     [0] = "REQUEST_POOL=impreports",
>>>> >> >   },
>>>> >> >   04: hadoop_user (string) = "fawzea",
>>>> >> > }
>>>> >> > I0208 01:07:06.321281  7154 impala-beeswax-server.cc:459]
>>>> >> > TClientRequest.queryOptions: TQueryOptions {
>>>> >> >   01: abort_on_error (bool) = false,
>>>> >> >   02: max_errors (i32) = 100,
>>>> >> >   03: disable_codegen (bool) = false,
>>>> >> >   04: batch_size (i32) = 0,
>>>> >> >   05: num_nodes (i32) = 0,
>>>> >> >   06: max_scan_range_length (i64) = 0,
>>>> >> >   07: num_scanner_threads (i32) = 0,
>>>> >> >   08: max_io_buffers (i32) = 0,
>>>> >> >   09: allow_unsupported_formats (bool) = false,
>>>> >> >   10: default_order_by_limit (i64) = -1,
>>>> >> >   11: debug_action (string) = "",
>>>> >> >   12: mem_limit (i64) = 0,
>>>> >> >   13: abort_on_default_limit_exceeded (bool) = false,
>>>> >> >   15: hbase_caching (i32) = 0,
>>>> >> >   16: hbase_cache_blocks (bool) = false,
>>>> >> >   17: parquet_file_size (i64) = 0,
>>>> >> >   18: explain_level (i32) = 1,
>>>> >> >   19: sync_ddl (bool) = false,
>>>> >> >   20: request_pool (string) = "impreports",
>>>> >> >   23: disable_cached_reads (bool) = false,
>>>> >> >   24: disable_outermost_topn (bool) = false,
>>>> >> >   25: rm_initial_mem (i64) = 0,
>>>> >> >   26: query_timeout_s (i32) = 0,
>>>> >> >   28: appx_count_distinct (bool) = false,
>>>> >> >   29: disable_unsafe_spills (bool) = false,
>>>> >> >   31: exec_single_node_rows_threshold (i32) = 100,
>>>> >> >   32: optimize_partition_key_scans (bool) = false,
>>>> >> >   33: replica_preference (i32) = 0,
>>>> >> >   34: schedule_random_replica (bool) = false,
>>>> >> >   35: scan_node_codegen_threshold (i64) = 1800000,
>>>> >> >   36: disable_streaming_preaggregations (bool) = false,
>>>> >> >   37: runtime_filter_mode (i32) = 2,
>>>> >> >   38: runtime_bloom_filter_size (i32) = 1048576,
>>>> >> >   39: runtime_filter_wait_time_ms (i32) = 0,
>>>> >> >   40: disable_row_runtime_filtering (bool) = false,
>>>> >> >   41: max_num_runtime_filters (i32) = 10,
>>>> >> >   42: parquet_annotate_strings_utf8 (bool) = false,
>>>> >> >   43: parquet_fallback_schema_resolution (i32) = 0,
>>>> >> >   45: s3_skip_insert_staging (bool) = true,
>>>> >> >   46: runtime_filter_min_size (i32) = 1048576,
>>>> >> >   47: runtime_filter_max_size (i32) = 16777216,
>>>> >> >   48: prefetch_mode (i32) = 1,
>>>> >> >   49: strict_mode (bool) = false,
>>>> >> >   50: scratch_limit (i64) = -1,
>>>> >> >   51: enable_expr_rewrites (bool) = true,
>>>> >> >   52: decimal_v2 (bool) = false,
>>>> >> >   53: parquet_dictionary_filtering (bool) = true,
>>>> >> >   54: parquet_array_resolution (i32) = 2,
>>>> >> >   55: parquet_read_statistics (bool) = true,
>>>> >> >   56: default_join_distribution_mode (i32) = 0,
>>>> >> >   57: disable_codegen_rows_threshold (i32) = 50000,
>>>> >> >   58: default_spillable_buffer_size (i64) = 2097152,
>>>> >> >   59: min_spillable_buffer_size (i64) = 65536,
>>>> >> >   60: max_row_size (i64) = 524288,
>>>> >> > }
>>>> >> > I0208 01:07:06.324410  7154 Frontend.java:909] Compiling query:
>>>> select
>>>> >> > count(*) from dv_entersiteevent_prq_remote where year=2017 and
>>>> month=10
>>>> >> > I0208 01:07:06.327203  7154 Frontend.java:948] Compiled query.
>>>> >> > I0208 01:07:06.359748  7154 admission-controller.cc:510] Schedule
>>>> for
>>>> >> > id=5f49e927591a9042:faf8f28600000000 in pool_name=root.impreports
>>>> >> > cluster_mem_needed=105.00 MB PoolConfig: max_requests=-1
>>>> max_queued=200
>>>> >> > max_mem=-1.00 B
>>>> >> > I0208 01:07:06.359776  7154 admission-controller.cc:515] Stats:
>>>> >> > agg_num_running=0, agg_num_queued=0, agg_mem_reserved=0,
>>>> >> > local_host(local_mem_admitted=0, num_admitted_running=0,
>>>> num_queued=0,
>>>> >> > backend_mem_reserved=0)
>>>> >> > I0208 01:07:06.359784  7154 admission-controller.cc:531] Admitted
>>>> query
>>>> >> > id=5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.359807  7154 coordinator.cc:99] Exec()
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000 stmt=select count(*)
>>>> from
>>>> >> > dv_entersiteevent_prq_remote where year=2017 and month=10
>>>> >> > I0208 01:07:06.360694  7154 coordinator.cc:357] starting execution
>>>> on 5
>>>> >> > backends for query 5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.363255 63583 impala-internal-service.cc:44]
>>>> >> > ExecQueryFInstances(): query_id=5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.363282 63583 query-exec-mgr.cc:46]
>>>> StartQueryFInstances()
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000
>>>> >> > coord=aopr-dhc001.lpdomain.com:22000
>>>> >> > I0208 01:07:06.363296 63583 query-state.cc:173] Buffer pool limit
>>>> for
>>>> >> > 5f49e927591a9042:faf8f28600000000: 10802010521
>>>> >> > I0208 01:07:06.363364 63583 initial-reservations.cc:60]
>>>> Successfully
>>>> >> > claimed
>>>> >> > initial reservations (0) for query 5f49e927591a9042:faf8f28600000
>>>> 000
>>>> >> > I0208 01:07:06.363544  7155 query-state.cc:286] StartFInstances():
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000 #instances=2
>>>> >> > I0208 01:07:06.363844  7155 query-state.cc:299] descriptor table
>>>> for
>>>> >> > query=5f49e927591a9042:faf8f28600000000
>>>> >> > tuples:
>>>> >> > Tuple(id=2 size=0 slots=[] tuple_path=[])
>>>> >> > Tuple(id=1 size=8 slots=[Slot(id=2 type=BIGINT col_path=[] offset=0
>>>> >> > null=(offset=0 mask=0) slot_idx=0 field_idx=-1)] tuple_path=[])
>>>> >> > Tuple(id=0 size=8 slots=[Slot(id=3 type=BIGINT col_path=[] offset=0
>>>> >> > null=(offset=0 mask=0) slot_idx=0 field_idx=-1)] tuple_path=[])
>>>> >> > I0208 01:07:06.364013  7156 query-state.cc:377] Executing instance.
>>>> >> > instance_id=5f49e927591a9042:faf8f28600000000 fragment_idx=0
>>>> >> > per_fragment_instance_idx=0 coord_state_idx=0 #in-flight=1
>>>> >> > I0208 01:07:06.364135  7157 query-state.cc:377] Executing instance.
>>>> >> > instance_id=5f49e927591a9042:faf8f28600000005 fragment_idx=1
>>>> >> > per_fragment_instance_idx=4 coord_state_idx=0 #in-flight=2
>>>> >> > I0208 01:07:06.364241  7157 hdfs-scan-node.cc:160] Max row batch
>>>> queue
>>>> >> > size
>>>> >> > for scan node '0' in fragment instance
>>>> >> > '5f49e927591a9042:faf8f28600000005':
>>>> >> > 160
>>>> >> > I0208 01:07:06.366775  7154 coordinator.cc:370] started execution
>>>> on 5
>>>> >> > backends for query 5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.385625  7155 query-exec-mgr.cc:149]
>>>> ReleaseQueryState():
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=4
>>>> >> > I0208 01:07:06.671193  7559 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.16.136.3:55598 closed, closing 1 associated session(s)
>>>> >> > I0208 01:07:06.676138   544 data-stream-mgr.cc:228] Reduced stream
>>>> ID
>>>> >> > cache
>>>> >> > from 2 items, to 0, eviction took: 0
>>>> >> > I0208 01:07:06.677202   544 coordinator.cc:955] Backend completed:
>>>> >> > host=aopr-dhc002.lpdomain.com:22000 remaining=4
>>>> >> > I0208 01:07:06.677214   544 coordinator-backend-state.cc:229]
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress
>>>> backend:
>>>> >> > aopr-dhc001.lpdomain.com:22000
>>>> >> > I0208 01:07:06.684221  7157 query-state.cc:384] Instance completed.
>>>> >> > instance_id=5f49e927591a9042:faf8f28600000005 #in-flight=1
>>>> status=OK
>>>> >> > I0208 01:07:06.684237  7157 query-exec-mgr.cc:149]
>>>> ReleaseQueryState():
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=3
>>>> >> > I0208 01:07:06.700275   520 coordinator.cc:955] Backend completed:
>>>> >> > host=aopr-dhc005.lpdomain.com:22000 remaining=3
>>>> >> > I0208 01:07:06.700289   520 coordinator-backend-state.cc:229]
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress
>>>> backend:
>>>> >> > aopr-dhc001.lpdomain.com:22000
>>>> >> > I0208 01:07:06.704620 65258 coordinator.cc:955] Backend completed:
>>>> >> > host=aopr-dhc003.lpdomain.com:22000 remaining=2
>>>> >> > I0208 01:07:06.704666 65258 coordinator-backend-state.cc:229]
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress
>>>> backend:
>>>> >> > aopr-dhc001.lpdomain.com:22000
>>>> >> > I0208 01:07:06.706041  7156 data-stream-mgr.cc:238]
>>>> DeregisterRecvr():
>>>> >> > fragment_instance_id=5f49e927591a9042:faf8f28600000000, node=2
>>>> >> > I0208 01:07:06.706070  7156 data-stream-recvr.cc:235] cancelled
>>>> stream:
>>>> >> > fragment_instance_id_=5f49e927591a9042:faf8f28600000000 node_id=2
>>>> >> > I0208 01:07:06.706809 64916 coordinator.cc:955] Backend completed:
>>>> >> > host=aopr-dhc004.lpdomain.com:22000 remaining=1
>>>> >> > I0208 01:07:06.706821 64916 coordinator-backend-state.cc:229]
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000: first in-progress
>>>> backend:
>>>> >> > aopr-dhc001.lpdomain.com:22000
>>>> >> > I0208 01:07:06.789568  7154 impala-beeswax-server.cc:193]
>>>> >> > get_results_metadata(): query_id=5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.790439  7154 coordinator.cc:789] Coordinator
>>>> waiting for
>>>> >> > backends to finish, 1 remaining
>>>> >> > I0208 01:07:06.791052  7154 coordinator.cc:794] All backends
>>>> finished
>>>> >> > successfully.
>>>> >> > I0208 01:07:06.791086  7154 coordinator.cc:1090] Release admssion
>>>> >> > control
>>>> >> > resources for query 5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.791363  7156 query-state.cc:384] Instance completed.
>>>> >> > instance_id=5f49e927591a9042:faf8f28600000000 #in-flight=0
>>>> status=OK
>>>> >> > I0208 01:07:06.791414  7156 query-exec-mgr.cc:149]
>>>> ReleaseQueryState():
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=2
>>>> >> > I0208 01:07:06.793586  7154 impala-beeswax-server.cc:235] close():
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.793602  7154 impala-server.cc:992]
>>>> UnregisterQuery():
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.793614  7154 impala-server.cc:1075] Cancel():
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.793624  7154 coordinator.cc:895] Cancel()
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000
>>>> >> > I0208 01:07:06.793637  7154 coordinator.cc:905] CancelBackends()
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000, tried to cancel 0
>>>> backends
>>>> >> > I0208 01:07:06.802776  7154 query-exec-mgr.cc:149]
>>>> ReleaseQueryState():
>>>> >> > query_id=5f49e927591a9042:faf8f28600000000 refcnt=1
>>>> >> > I0208 01:07:06.814214  7154 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.16.144.131:51012 closed, closing 1 associated session(s)
>>>> >> > I0208 01:07:07.258891  7567 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.16.136.2:56375 closed, closing 1 associated session(s)
>>>> >> > I0208 01:07:11.600131  7819 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.16.136.3:27460 closed, closing 1 associated session(s)
>>>> >> > I0208 01:07:12.262739  7821 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.16.136.2:52672 closed, closing 1 associated session(s)
>>>> >> > I0208 01:07:16.665451  7822 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.16.136.3:10693 closed, closing 1 associated session(s)
>>>> >> > I0208 01:07:17.235321  7825 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.16.136.2:34752 closed, closing 1 associated session(s)
>>>> >> > I0208 01:07:21.410611  5858 impala-beeswax-server.cc:193]
>>>> >> > get_results_metadata(): query_id=1d4fbf0f2571f1e9:ea346f7900000000
>>>> >> > I0208 01:07:21.414175  5858 impala-beeswax-server.cc:235] close():
>>>> >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000
>>>> >> > I0208 01:07:21.414185  5858 impala-server.cc:992]
>>>> UnregisterQuery():
>>>> >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000
>>>> >> > I0208 01:07:21.414191  5858 impala-server.cc:1075] Cancel():
>>>> >> > query_id=1d4fbf0f2571f1e9:ea346f7900000000
>>>> >> > I0208 01:07:21.421505  5858 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.26.180.75:60072 closed, closing 1 associated session(s)
>>>> >> > I0208 01:07:21.605690  7830 impala-server.cc:1796] Connection from
>>>> >> > client
>>>> >> > ::ffff:10.16.136.3:46660 closed, closing 1 associated session(s)
>>>> >> >
>>>> >> >
>>>> >> > On Thu, Feb 8, 2018 at 6:20 AM, Matthew Jacobs <mjacobs@apache.org
>>>> >
>>>> >> > wrote:
>>>> >> >>
>>>> >> >> Thanks! I think that configuration looks OK.
>>>> >> >>
>>>> >> >> 1) What version of Impala is this?
>>>> >> >>
>>>> >> >> 2) Can you send the query profile and the Impala coordinator log
>>>> for
>>>> >> >> the query where it's not behaving as expected?
>>>> >> >>
>>>> >> >> > When i'm running the following with fawzea user:
>>>> >> >> > set request_pool=impreports;
>>>> >> >> > select count(*) from dv_entersiteevent_prq_remote where
>>>> year=2017 and
>>>> >> >> > month=10;
>>>> >> >> > I can see that the query is running with impreports pool.
>>>> >> >>
>>>> >> >> On Wed, Feb 7, 2018 at 7:59 PM, Fawze Abujaber <fawzeaj@gmail.com
>>>> >
>>>> >> >> wrote:
>>>> >> >> > Hi Matt,
>>>> >> >> >
>>>> >> >> > [root@aopr-dhc001 ~]# cat
>>>> >> >> >
>>>> >> >> >
>>>> >> >> > /var/run/cloudera-scm-agent/process/4558-impala-IMPALAD/impa
>>>> la-conf/fair-scheduler.xml
>>>> >> >> > <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
>>>> >> >> > <allocations>
>>>> >> >> >     <queue name="root">
>>>> >> >> >         <weight>1.0</weight>
>>>> >> >> >         <schedulingPolicy>fair</schedulingPolicy>
>>>> >> >> >         <queue name="impiaas">
>>>> >> >> >             <weight>1.0</weight>
>>>> >> >> >             <schedulingPolicy>fair</schedulingPolicy>
>>>> >> >> >             <aclSubmitApps>fawzea </aclSubmitApps>
>>>> >> >> >         </queue>
>>>> >> >> >         <queue name="impreports">
>>>> >> >> >             <weight>1.0</weight>
>>>> >> >> >             <schedulingPolicy>fair</schedulingPolicy>
>>>> >> >> >             <aclSubmitApps>nira </aclSubmitApps>
>>>> >> >> >         </queue>
>>>> >> >> >     </queue>
>>>> >> >> >     <defaultQueueSchedulingPolicy>fair</defaultQueueSchedulingP
>>>> olicy>
>>>> >> >> >     <queuePlacementPolicy>
>>>> >> >> >         <rule name="specified" create="false"/>
>>>> >> >> >         <rule name="secondaryGroupExistingQueue"/>
>>>> >> >> >         <rule name="default"/>
>>>> >> >> >     </queuePlacementPolicy>
>>>> >> >> > </allocations>
>>>> >> >> >
>>>> >> >> >
>>>> >> >> > On Thu, Feb 8, 2018 at 4:47 AM, Matthew Jacobs <
>>>> jacobs.mh@gmail.com>
>>>> >> >> > wrote:
>>>> >> >> >>
>>>> >> >> >> Yes, I saw that, but I'd like to see the actual configuration
>>>> that
>>>> >> >> >> is
>>>> >> >> >> passed to Impala because it may not be specified in the right
>>>> way to
>>>> >> >> >> accomplish what you're trying to do in "b".
>>>> >> >> >>
>>>> >> >> >> On Wed, Feb 7, 2018 at 4:32 PM, Fawze Abujaber <
>>>> fawzeaj@gmail.com>
>>>> >> >> >> wrote:
>>>> >> >> >> > Hi Matt,
>>>> >> >> >> >
>>>> >> >> >> > I created the pools manually and it’s completely
>>>> nfigurations as
>>>> >> >> >> > shown
>>>> >> >> >> > in b.
>>>> >> >> >> >
>>>> >> >> >> > On Wed, 7 Feb 2018 at 23:17 Matthew Jacobs <
>>>> jacobs.mh@gmail.com>
>>>> >> >> >> > wrote:
>>>> >> >> >> >>
>>>> >> >> >> >> How, more specifically, are you configuring these pools?
>>>> Can you
>>>> >> >> >> >> share
>>>> >> >> >> >> the configuration files?
>>>> >> >> >> >>
>>>> >> >> >> >> On Wed, Feb 7, 2018 at 11:42 AM, Fawze Abujaber
>>>> >> >> >> >> <fawzeaj@gmail.com>
>>>> >> >> >> >> wrote:
>>>> >> >> >> >> > Hello All,
>>>> >> >> >> >> >
>>>> >> >> >> >> > Will appreciate any insight of what i'm missing in my
>>>> >> >> >> >> > configuration
>>>> >> >> >> >> > to
>>>> >> >> >> >> > get
>>>> >> >> >> >> > the impala submission control working as expected ....
>>>> >> >> >> >> >
>>>> >> >> >> >> > a- Here is my placement Placement Rule:
>>>> >> >> >> >> > 1- Use the pool Specified at run time., only if the pool
>>>> >> >> >> >> > exists.
>>>> >> >> >> >> > 2- Use the pool root.[secondary group].
>>>> >> >> >> >> > 3- Use the pool root.default.
>>>> >> >> >> >> >
>>>> >> >> >> >> > b- Here is the impala resource pools
>>>> >> >> >> >> >
>>>> >> >> >> >> > Name                    Max Memory Max Running Queries Max
>>>> >> >> >> >> > Queued
>>>> >> >> >> >> > Queries
>>>> >> >> >> >> > Queue Timeout Default Query Memory Limit Access Control
>>>> >> >> >> >> > root.impiaas            Unlimited
>>>>  Unlimited
>>>> >> >> >> >> > 200                          1 minute(s)            No
>>>> Default
>>>> >> >> >> >> > Custom
>>>> >> >> >> >> > root.impreports        Unlimited
>>>>  Unlimited
>>>> >> >> >> >> > 200                          1 minute(s)            No
>>>> Default
>>>> >> >> >> >> > Custom
>>>> >> >> >> >> >
>>>> >> >> >> >> >
>>>> >> >> >> >> > C- Here is my Access control for root.impiaas pool:
>>>> >> >> >> >> >
>>>> >> >> >> >> > Allow these users and groups to submit to this pool
>>>> >> >> >> >> >
>>>> >> >> >> >> > Users: fawzea
>>>> >> >> >> >> >
>>>> >> >> >> >> > The same for impreports the allowed user and groups to
>>>> submit
>>>> >> >> >> >> > to
>>>> >> >> >> >> > pool
>>>> >> >> >> >> > is
>>>> >> >> >> >> > only nira.
>>>> >> >> >> >> >
>>>> >> >> >> >> >
>>>> >> >> >> >> > LDAP groups for fawzea user:
>>>> >> >> >> >> >
>>>> >> >> >> >> >  Member in groups:
>>>> >> >> >> >> >
>>>> >> >> >> >> >  - dba
>>>> >> >> >> >> >  - impiaas
>>>> >> >> >> >> >  - mysql
>>>> >> >> >> >> >
>>>> >> >> >> >> >  LDAP group for nira user
>>>> >> >> >> >> >   Member in groups:
>>>> >> >> >> >> >  - dba
>>>> >> >> >> >> >  - impreports
>>>> >> >> >> >> >  - mysql
>>>> >> >> >> >> >
>>>> >> >> >> >> >
>>>> >> >> >> >> >
>>>> >> >> >> >> > Impala Queries UI shows:
>>>> >> >> >> >> >
>>>> >> >> >> >> > 1- select count(*) from dv_entersiteevent_prq_remote where
>>>> >> >> >> >> > year=2017
>>>> >> >> >> >> >
>>>> >> >> >> >> > User: fawzea
>>>> >> >> >> >> > Pool: root.impiaas
>>>> >> >> >> >> >
>>>> >> >> >> >> > 2- select count(*) from dv_entersiteevent_prq_remote where
>>>> >> >> >> >> > year=2017
>>>> >> >> >> >> > and
>>>> >> >> >> >> > month=10
>>>> >> >> >> >> > User: nira
>>>> >> >> >> >> > Pool: root.impreports
>>>> >> >> >> >> >
>>>> >> >> >> >> > As shown above that the queries submitted based on the
>>>> >> >> >> >> > secondary
>>>> >> >> >> >> > group
>>>> >> >> >> >> > rule.
>>>> >> >> >> >> >
>>>> >> >> >> >> > When i'm running the following with fawzea user:
>>>> >> >> >> >> >
>>>> >> >> >> >> > set request_pool=impreports;
>>>> >> >> >> >> > select count(*) from dv_entersiteevent_prq_remote where
>>>> >> >> >> >> > year=2017
>>>> >> >> >> >> > and
>>>> >> >> >> >> > month=10;
>>>> >> >> >> >> >
>>>> >> >> >> >> > I can see that the query is running with impreports pool.
>>>> >> >> >> >> >
>>>> >> >> >> >> >
>>>> >> >> >> >> > What i'm missing here in my configuration to get this
>>>> works as
>>>> >> >> >> >> > expected.
>>>> >> >> >> >> >
>>>> >> >> >
>>>> >> >> >
>>>> >> >
>>>> >> >
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>

Mime
View raw message