hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "huaxiang sun (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-19994) Create a new class for RPC throttling exception, make it retryable.
Date Tue, 13 Feb 2018 22:32:00 GMT
huaxiang sun created HBASE-19994:

             Summary: Create a new class for RPC throttling exception, make it retryable.

                 Key: HBASE-19994
                 URL: https://issues.apache.org/jira/browse/HBASE-19994
             Project: HBase
          Issue Type: Improvement
            Reporter: huaxiang sun

Based on a discussion at dev mailing list.


Thanks Andrew.

+1 for the second option, I will create a jira for this change.


On Feb 9, 2018, at 1:09 PM, Andrew Purtell <apurtell@apache.org> wrote:

We have

public class ThrottlingException extends QuotaExceededException

public class QuotaExceededException extends DoNotRetryIOException

Let the storage quota limits throw QuotaExceededException directly (based

on DNRIOE). That seems fine.

However, ThrottlingException is thrown as a result of a temporal quota,

so it is inappropriate for this to inherit from DNRIOE, it should inherit

IOException instead so the client is allowed to retry until successful, or

until the retry policy is exhausted.

We are in a bit of a pickle because we've released with this inheritance

hierarchy, so to change it we will need a new minor, or we will want to

deprecate ThrottlingException and use a new exception class instead, one

which does not inherit from DNRIOE.

On Feb 7, 2018, at 9:25 AM, Huaxiang Sun <hsun@cloudera.com> wrote:

Hi Mike,

  You are right. For rpc throttling, definitely it is retryable. For storage quota, I think
it will be fail faster (non-retryable).

  We probably need to separate these two types of exceptions, I will do some more research
and follow up.



On Feb 7, 2018, at 9:16 AM, Mike Drob <mdrob@apache.org> wrote:

I think, philosophically, there can be two kinds of QEE -

For throttling, we can retry. The quota is a temporal quota - you have done

too many operations this minute, please try again next minute and

everything will work.

For storage, we shouldn't retry. The quota is a fixed quote - you have

exceeded your allotted disk space, please do not try again until you have

remedied the situation.

Our current usage conflates the two, sometimes it is correct, sometimes not.

On Wed, Feb 7, 2018 at 11:00 AM, Huaxiang Sun <hsun@cloudera.com> wrote:

Hi Stack,

 I run into a case that a mapreduce job in hive cannot finish because

it runs into a QEE.

I need to look into the hive mr task to see if QEE is not handled

correctly in hbase code or in hive code.

I am thinking that if  QEE is a retryable exception, then it should be

taken care of by the hbase code.

I will check more and report back.



On Feb 7, 2018, at 8:23 AM, Stack <stack@duboce.net> wrote:

QEE being a DNRIOE seems right on the face of it.

But if throttling, a DNRIOE is inappropriate. Where you seeing a QEE in a

throttling scenario Huaxiang?




This message was sent by Atlassian JIRA

View raw message