impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fawze Abujaber <fawz...@gmail.com>
Subject Re: impala submission control
Date Thu, 08 Feb 2018 03:59:00 GMT
Hi Matt,

[root@aopr-dhc001 ~]# cat
/var/run/cloudera-scm-agent/process/4558-impala-IMPALAD/impala-conf/fair-scheduler.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<allocations>
    <queue name="root">
        <weight>1.0</weight>
        <schedulingPolicy>fair</schedulingPolicy>
        <queue name="impiaas">
            <weight>1.0</weight>
            <schedulingPolicy>fair</schedulingPolicy>
            <aclSubmitApps>fawzea </aclSubmitApps>
        </queue>
        <queue name="impreports">
            <weight>1.0</weight>
            <schedulingPolicy>fair</schedulingPolicy>
            <aclSubmitApps>nira </aclSubmitApps>
        </queue>
    </queue>
    <defaultQueueSchedulingPolicy>fair</defaultQueueSchedulingPolicy>
    <queuePlacementPolicy>
        <rule name="specified" create="false"/>
        <rule name="secondaryGroupExistingQueue"/>
        <rule name="default"/>
    </queuePlacementPolicy>
</allocations>


On Thu, Feb 8, 2018 at 4:47 AM, Matthew Jacobs <jacobs.mh@gmail.com> wrote:

> Yes, I saw that, but I'd like to see the actual configuration that is
> passed to Impala because it may not be specified in the right way to
> accomplish what you're trying to do in "b".
>
> On Wed, Feb 7, 2018 at 4:32 PM, Fawze Abujaber <fawzeaj@gmail.com> wrote:
> > Hi Matt,
> >
> > I created the pools manually and it’s completely nfigurations as shown
> in b.
> >
> > On Wed, 7 Feb 2018 at 23:17 Matthew Jacobs <jacobs.mh@gmail.com> wrote:
> >>
> >> How, more specifically, are you configuring these pools? Can you share
> >> the configuration files?
> >>
> >> On Wed, Feb 7, 2018 at 11:42 AM, Fawze Abujaber <fawzeaj@gmail.com>
> wrote:
> >> > Hello All,
> >> >
> >> > Will appreciate any insight of what i'm missing in my configuration to
> >> > get
> >> > the impala submission control working as expected ....
> >> >
> >> > a- Here is my placement Placement Rule:
> >> > 1- Use the pool Specified at run time., only if the pool exists.
> >> > 2- Use the pool root.[secondary group].
> >> > 3- Use the pool root.default.
> >> >
> >> > b- Here is the impala resource pools
> >> >
> >> > Name                    Max Memory Max Running Queries Max Queued
> >> > Queries
> >> > Queue Timeout Default Query Memory Limit Access Control
> >> > root.impiaas            Unlimited                   Unlimited
> >> > 200                          1 minute(s)            No Default
> >> > Custom
> >> > root.impreports        Unlimited                   Unlimited
> >> > 200                          1 minute(s)            No Default
> >> > Custom
> >> >
> >> >
> >> > C- Here is my Access control for root.impiaas pool:
> >> >
> >> > Allow these users and groups to submit to this pool
> >> >
> >> > Users: fawzea
> >> >
> >> > The same for impreports the allowed user and groups to submit to pool
> is
> >> > only nira.
> >> >
> >> >
> >> > LDAP groups for fawzea user:
> >> >
> >> >  Member in groups:
> >> >
> >> >  - dba
> >> >  - impiaas
> >> >  - mysql
> >> >
> >> >  LDAP group for nira user
> >> >   Member in groups:
> >> >  - dba
> >> >  - impreports
> >> >  - mysql
> >> >
> >> >
> >> >
> >> > Impala Queries UI shows:
> >> >
> >> > 1- select count(*) from dv_entersiteevent_prq_remote where year=2017
> >> >
> >> > User: fawzea
> >> > Pool: root.impiaas
> >> >
> >> > 2- select count(*) from dv_entersiteevent_prq_remote where year=2017
> and
> >> > month=10
> >> > User: nira
> >> > Pool: root.impreports
> >> >
> >> > As shown above that the queries submitted based on the secondary group
> >> > rule.
> >> >
> >> > When i'm running the following with fawzea user:
> >> >
> >> > set request_pool=impreports;
> >> > select count(*) from dv_entersiteevent_prq_remote where year=2017 and
> >> > month=10;
> >> >
> >> > I can see that the query is running with impreports pool.
> >> >
> >> >
> >> > What i'm missing here in my configuration to get this works as
> expected.
> >> >
>

Mime
View raw message