hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Huaxiang Sun <h...@cloudera.com>
Subject Re: QuotaExceededException as a DoNotRetryIOException?
Date Wed, 07 Feb 2018 17:25:21 GMT
Hi Mike,

    You are right. For rpc throttling, definitely it is retryable. For storage quota, I think
it will be fail faster (non-retryable).
    We probably need to separate these two types of exceptions, I will do some more research
and follow up.

    Thanks,
    Huaxiang

> On Feb 7, 2018, at 9:16 AM, Mike Drob <mdrob@apache.org> wrote:
> 
> I think, philosophically, there can be two kinds of QEE -
> 
> For throttling, we can retry. The quota is a temporal quota - you have done
> too many operations this minute, please try again next minute and
> everything will work.
> For storage, we shouldn't retry. The quota is a fixed quote - you have
> exceeded your allotted disk space, please do not try again until you have
> remedied the situation.
> 
> Our current usage conflates the two, sometimes it is correct, sometimes not.
> 
> On Wed, Feb 7, 2018 at 11:00 AM, Huaxiang Sun <hsun@cloudera.com> wrote:
> 
>> Hi Stack,
>> 
>>    I run into a case that a mapreduce job in hive cannot finish because
>> it runs into a QEE.
>> I need to look into the hive mr task to see if QEE is not handled
>> correctly in hbase code or in hive code.
>> 
>>   I am thinking that if  QEE is a retryable exception, then it should be
>> taken care of by the hbase code.
>>   I will check more and report back.
>> 
>>   Thanks,
>>   Huaxiang
>> 
>>> On Feb 7, 2018, at 8:23 AM, Stack <stack@duboce.net> wrote:
>>> 
>>> QEE being a DNRIOE seems right on the face of it.
>>> 
>>> But if throttling, a DNRIOE is inappropriate. Where you seeing a QEE in a
>>> throttling scenario Huaxiang?
>>> 
>>> Thanks,
>>> S
>>> 
>>> 
>>> On Tue, Feb 6, 2018 at 4:56 PM, Huaxiang Sun <hsun@cloudera.com> wrote:
>>> 
>>>> Hi HBase devs,
>>>> 
>>>>   I found that QuotaExceededException  is a DoNotRetryIOException,
>> which
>>>> is a bit strange from user’s point of view.
>>>>   For rpc throttling, the exception is retryable and it tells app to
>>>> slow down and retry later.
>>>> 
>>>>   Any thoughts?
>>>> 
>>>>   Thanks,
>>>>   Huaxiang
>> 
>> 


Mime
View raw message