hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: Scan priority based on type of request origin
Date Mon, 01 Jun 2015 10:21:12 GMT
That is right.  So if your implementation is just based on the priority of
the user then the queue will be fetching the request coming out of a higher
priority user always.

If there are multiple requests from the higher priority user/users - how
will you distinguish between those requests?  Within users of same priority
you will again categorise it based on time?

Regards
Ram

On Mon, Jun 1, 2015 at 3:11 PM, ashish singhi <ashish.singhi@huawei.com>
wrote:

> @Ram:
> > I would say the deadline based impl would queue up the one with 0
> deadline as the prioritized one rather than the normal one.
> No. The actual deadline also includes the current timestamp. So the
> request which has before timestamp will be first in the queue in this case.
>
> > But if you want to implement any priority based scheduling then you can
> have a new scheduler and make your own queue implementation out of it.
> Yeah.
>
> Regards,
> Ashish Singhi
>
>
> -----Original Message-----
> From: ramkrishna vasudevan [mailto:ramkrishna.s.vasudevan@gmail.com]
> Sent: 28 May 2015 23:43
> To: dev@hbase.apache.org
> Subject: Re: Scan priority based on type of request origin
>
> bq.Consider a case when there is one normal priority request (submitted at
> t1) with deadline X in the queue and one more high priority request
> (Submitted at t2 >= t1 + x ) with zero deadline. In the current
> implementation the normal priority request will be offered from the queue
>
> I would say the deadline based impl would queue up the one with 0 deadline
> as the prioritized one rather than the normal one.  See this
> BoundedPriorityBlockingQueue.  Am I missing something here?
>
> But if you want to implement any priority based scheduling then you can
> have a new scheduler and make your own queue implementation out of it.
>
> On Thu, May 28, 2015 at 3:08 PM, ashish singhi <ashish.singhi@huawei.com>
> wrote:
>
> > Thanks for the quick responses.
> >
> > @Anoop:
> > Yes, we were also thinking of implementing it along the same lines.
> >
> > @Ram:
> > High priority request should be always given preference over all other
> > scan request.
> > As per my understanding in the current implementation of deadline the
> > scenario cannot be satisfied. Consider a case when there is one normal
> > priority request (submitted at t1) with deadline X in the queue and
> > one more high priority request (Submitted at t2 >= t1 + x ) with zero
> deadline.
> > In the current implementation the normal priority request will be
> > offered from the queue. But our scenario require high priority request
> > to be offered.  So there is a need of some more decision parameters
> > (such as
> > priority) in the Scheduler logic.
> > Please correct me If I am missing something.
> >
> > Vladimir Rodionov wrote
> > > Scanner prioritization is not sufficient, there are a lot of other
> > > IO
> > activity (compaction, flushing, logging), which can severely affect
> > high priority scanners.
> > >
> > > HBase IO scheduler anyone?
> >
> > Yes I agree, Scan priority alone is not sufficient enough but its
> > required at first place. With stuffs like IO scheduler the solution will
> be complete.
> >
> > Regards,
> > Ashish Singhi
> >
> >
> > -----Original Message-----
> > From: ramkrishna vasudevan [mailto:ramkrishna.s.vasudevan@gmail.com]
> > Sent: 28 May 2015 12:18
> > To: dev@hbase.apache.org
> > Subject: Re: Scan priority based on type of request origin
> >
> > Atleast in trunk it uses SimpleRPCScheduler and internally it tries to
> > use the Priority based queue so that based on the deadline set for
> > every request it can schedule the highest priority RPC calls.
> >
> > I would think that your current requirement may fit with the existing
> one?
> > Except that those calls should be given higher prioirty or rather a
> > deadline such it is scheduled first?
> >
> > On Thu, May 28, 2015 at 12:14 PM, Anoop John <anoop.hbase@gmail.com>
> > wrote:
> >
> > > We have pluggable RPC scheduler now.  See config
> > > 'hbase.region.server.rpc.scheduler.factory.class'
> > >
> > > By default a SimpleRpcScheduler is in use and there as you said,
> > > there are prioritization for super user.  Other prioritization is
> > > based on the RS side method prioritization based on annotation.
> > >
> > > You can see how/whether u can have a custom RPC scheduler and
> > > queuing mechanism using which you can fulfill this.
> > >
> > > -Anoop-
> > >
> > > On Thu, May 28, 2015 at 11:56 AM, ashish singhi
> > > <ashish.singhi@huawei.com>
> > > wrote:
> > >
> > > > Hello folks.
> > > >
> > > > We had a customer scenario to support scan priority based on type
> > > > of request origin. Consider the scenario , where in n number of
> > > > scan
> > > requests
> > > > are generated from different clients. Some of these requests are
> > > > high priority as they are originated from high privileged users OR
> > > > from data viewer Web UI etc. Such request should not wait and must
> > > > finish execution at the earliest compare to other requests.
> > > >
> > > > We went through the code and found that HBase support for
> > > > de-prioritizing long running scanner(HBASE-10993) and providing
> > > > super user rpc requests higher priority comparing to other
> > > > users(HBASE-13375). So this means
> > > apart
> > > > from super user all other user request will be treated as same and
> > > > with normal QOS.
> > > >
> > > > Let us know if there are any existing solutions already available
> > > > which may be we are missing.
> > > >
> > > > If implementing this make sense , we can implement this in community.
> > > >
> > > > Regards,
> > > > Ashish Singhi
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message