hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-9673) Request priority revamp
Date Fri, 27 Sep 2013 23:12:03 GMT
stack created HBASE-9673:

             Summary: Request priority revamp
                 Key: HBASE-9673
                 URL: https://issues.apache.org/jira/browse/HBASE-9673
             Project: HBase
          Issue Type: Brainstorming
            Reporter: stack

Currently in the server we have three (four?) queues ranging from Normal (0) to High priority
(100) with Replication priority falling between the two.

Priority is determined by introspecting the incoming invocation to pull out which region is
targeted by the query and/or to read a 'priority' annotation that is on the invocation's method
-- e.g. closeRegion/openRegion have @QosPriority(priority=HConstants.HIGH_QOS)  These annotations
are awkwardly on the java implementation, not on the protobuf Service.

The current means of introspection determining priority running the silly guava function is
costly done on each call.  The annotations being on the java implementation rather than on
the pb Service or in the pbs themselves requires somersaults extracting priority. If they
were on the latter, then they would be more readily available down in rpc where invocation/scheduling
is done.

There is no way currently for a mapreduce task to tell hbase it is a low priority query nor
for a webservice coming in the door to say I need a bit of info now -- and I don't care who
you push aside.

A suggestion of [~eclark]'s is that we just have the client set the priority.  This has much
merit.  We could keep it down low just do what the server currently does -- the introspection,
annotations -- on client side and have it then set the priority into the request header where
it will be readily available on the server when it is trying to figure what to do w/ the incoming
call (high, low, or in-between).

We could even expose this in our API.

Above is about revamp of how the server figures the static priority of a call (HBASE-9612
actually adds priority to the RpcController and into the RequestHeader).  Server could just
respect the client priority for now.

Later we'd want the server to adjust priorities based off what it thought of the user -- trusted
or an alien -- as well as factor in its current load.  An overloaded server might reject low
priority calls (Had a good chat on QoS w/ [~toffer] about QoS at the corner of Van Ness and
Market last night).

Later calls might come w/ an advertised SLA.  The server might reject calls it thought it
could not service in time rather than take on more load (based off history of similarly sized
queries, etc.).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message