Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAAT7MkpSKkfGm4kB8CigXuHnU0jEFwQmeTO1p6JLCKMi0i-iFw@mail.gmail.com>
References: 
 <CAHnpetRGgkw2VH4FTb5==QUfDXndhc9Pi-TiZnj1B_jgo5s1PQ@mail.gmail.com>
 <CAAT7MkpSKkfGm4kB8CigXuHnU0jEFwQmeTO1p6JLCKMi0i-iFw@mail.gmail.com>
From: Matteo Bertozzi <theo.bertozzi@gmail.com>
Date: Fri, 4 Sep 2015 09:18:51 -0700
Message-ID: 
 <CAFb6HiLkzJPqVGApPJdHRDD1CL=-6xitA8mJ+JXeP9OBU-P_2w@mail.gmail.com>
Subject: Re: Question about RPC queues
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=001a1137948e8c2eb8051eee4263

--001a1137948e8c2eb8051eee4263
Content-Type: text/plain; charset=UTF-8

scan is chunked in small pieces, it is not a single long operation.
what fifo vs deadline does is pushing the scan chunk back in the queue
if you have multiple operation with higher priority.

after N runs of scanner.next() you'll be pushed back following a
sqrt(numOfNext * weight)
http://blog.cloudera.com/wp-content/uploads/2014/11/hbase-multi-f2.png

so, the more next you call the further your scan chunk will be delayed.
but once your scan chunk is in the executor, there is no way (at the moment)
to eject that task, so if you have a single thread and you have an
hi-priority get
that get has to wait the scan chunk to be executed.

unless you have tons of requests to fill the queue
and a super-long scan (or you increase the weight to simulate that)
you'll probably not be able to notice any difference.

On Fri, Sep 4, 2015 at 9:05 AM, ramkrishna vasudevan <
ramkrishna.s.vasudevan@gmail.com> wrote:

> Hi Kulkarni
>
> Which version of HBase are you using?  Just to confirm are you running as
> super user only?  I think there are some meta handlers also with which
> super user calls are handled.
>
> From the UI are you able to monitor the handlers and see how many are
> active and what is the status of it.
>
> Regards
> Ram
>
> On Fri, Sep 4, 2015 at 7:42 PM, kulkarni.swarnim@gmail.com <
> kulkarni.swarnim@gmail.com> wrote:
>
> > Hello all,
> >
> > So I am trying to play around with de-prioritizing long running scanners
> > feature added with HBASE-10993. Essentially I wanted to see how
> "deadline"
> > queue prevents contention. This is what my did for my setup. Changed the
> > callqueue type to "fifo" and reduced the number of RPC handlers to 1 to
> > ensure that I have only one RPC queue in place. I am also testing on a
> > cluster with only one RS to again make sure that there is only one RPC
> > handler working. I confirmed that my settings are properly in place with
> > the following message in the hmaster logs after I restarted my cluster.
> >
> > "INFO org.apache.hadoop.hbase.ipc.SimpleRpcScheduler: Using fifo as user
> > call queue, count=1"
> >
> > I now kicked off a long running "scan" first and while the scan was
> running
> > kicked off another "get". My hope was that with fifo, we should see a
> > contention and the "get" would essentially hang until the "scan" finished
> > as it would be placed behind the "scan" request in the RPC queue. But I
> see
> > the gets still return quickly irrespective of whether the scan is running
> > or not.
> >
> > Is there something obvious that I am missing here to be able to reproduce
> > the contention?
> >
> > Thanks,
> > Swarnim
> >
>

--001a1137948e8c2eb8051eee4263--