hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16857) RateLimiter may fails during parallel scan execution
Date Mon, 17 Oct 2016 06:58:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15581436#comment-15581436
] 

Hadoop QA commented on HBASE-16857:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s {color} | {color:blue}
Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s {color} | {color:red}
HBASE-16857 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12833692/HBASE-16857.patch
|
| JIRA Issue | HBASE-16857 |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/4050/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> RateLimiter may fails during parallel scan execution
> ----------------------------------------------------
>
>                 Key: HBASE-16857
>                 URL: https://issues.apache.org/jira/browse/HBASE-16857
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.1.2
>         Environment: hbase.quota.enabled=true 
> hbase.quota.refresh.period=5000 
>            Reporter: Sergey Soldatov
>            Assignee: Sergey Soldatov
>         Attachments: HBASE-16857.patch
>
>
> Steps to reproduce using phoenix (that's the easiest way to run a lot of parallel scans):
> 1. Create table:
> {code}
> create table "abc" (id bigint not null primary key, name varchar) salt_buckets=50; 
> {code}
> 2. set quota from hbase shell:
> {code}
> set_quota TYPE => THROTTLE, TABLE => 'abc', LIMIT => '10G/sec' 
> {code}
> 3. in phoenix run 
> {code}
>  select * from "abc"; 
> {code}
> It will fail with ThrottlingException.
> Sometimes it requires to run it several times to reproduce.
> That happens because the logic in DefaultOperationQuota. First we run limiter.checkQuota
where we may change available to Long.MAX_VALUE, after that we run limiter.grabQuota where
we reduce available for 1000 (is it scan overhead or what?) and in close() we adding this
1000 back. 
> When number of parallel scans are executing there is a chance that one of the threads
run limiter.checkQuota right before the second run close(). We get overflow and as the result
available value becomes negative, so during the next check we just fail. 
> This behavior was introduced in HBASE-13686.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message