hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18133) Low-latency space quota size reports
Date Tue, 20 Jun 2017 22:28:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056581#comment-16056581
] 

Hadoop QA commented on HBASE-18133:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 43s {color} | {color:blue}
Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} |
{color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color}
| {color:green} The patch appears to include 12 new or modified test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 31s {color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 17s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s {color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s {color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 47s {color}
| {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 8s {color} | {color:red}
hbase-server in master has 12 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s {color} | {color:green}
master passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s {color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 15s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 6s {color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 33m 34s {color}
| {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5
2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 28s {color} | {color:red}
hbase-server generated 1 new + 12 unchanged - 0 fixed = 13 total (was 12) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s {color} |
{color:green} hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s {color} |
{color:green} hbase-hadoop2-compat generated 0 new + 1 unchanged - 1 fixed = 1 total (was
2) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s {color} |
{color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 21s {color} | {color:green}
hbase-hadoop-compat in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s {color} | {color:green}
hbase-hadoop2-compat in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 22s {color} | {color:red}
hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 32s {color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 79m 8s {color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
|  |  Nullcheck of HRegion.rsServices at line 2764 of value previously dereferenced in org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(WAL,
MonitoredTask, HRegion$PrepareFlushResult, Collection)  At HRegion.java:2764 of value previously
dereferenced in org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(WAL,
MonitoredTask, HRegion$PrepareFlushResult, Collection)  At HRegion.java:[line 2682] |
| Failed junit tests | hadoop.hbase.regionserver.TestMinVersions |
|   | hadoop.hbase.filter.TestFilterFromRegionSide |
|   | hadoop.hbase.io.hfile.TestScannerSelectionUsingKeyRange |
|   | hadoop.hbase.regionserver.TestKeepDeletes |
|   | hadoop.hbase.io.encoding.TestPrefixTree |
|   | hadoop.hbase.coprocessor.TestCoprocessorInterface |
|   | hadoop.hbase.regionserver.TestScanWithBloomError |
|   | hadoop.hbase.client.TestIntraRowPagination |
|   | hadoop.hbase.regionserver.TestBlocksScanned |
|   | hadoop.hbase.filter.TestMultipleColumnPrefixFilter |
|   | hadoop.hbase.regionserver.TestStoreFileRefresherChore |
|   | hadoop.hbase.regionserver.TestScanner |
|   | hadoop.hbase.regionserver.TestWALMonotonicallyIncreasingSeqId |
|   | hadoop.hbase.filter.TestInvocationRecordFilter |
|   | hadoop.hbase.filter.TestFilter |
|   | hadoop.hbase.filter.TestDependentColumnFilter |
|   | hadoop.hbase.regionserver.TestResettingCounters |
|   | hadoop.hbase.coprocessor.TestRegionObserverStacking |
|   | hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan |
|   | hadoop.hbase.regionserver.TestWideScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12873737/HBASE-18133.002.patch
|
| JIRA Issue | HBASE-18133 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  hbaseanti  checkstyle
 compile  |
| uname | Linux aff9a79f8183 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 5b485d1 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
|
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/artifact/patchprocess/new-findbugs-hbase-server.html
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/artifact/patchprocess/patch-unit-hbase-server.txt
|
| unit test logs |  https://builds.apache.org/job/PreCommit-HBASE-Build/7258/artifact/patchprocess/patch-unit-hbase-server.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/testReport/ |
| modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Low-latency space quota size reports
> ------------------------------------
>
>                 Key: HBASE-18133
>                 URL: https://issues.apache.org/jira/browse/HBASE-18133
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 3.0.0
>
>         Attachments: HBASE-18133.001.patch, HBASE-18133.002.patch
>
>
> Presently space quota enforcement relies on RegionServers sending reports to the master
about each Region that they host. This is done by periodically, reading the cached size of
each HFile in each Region (which was ultimately computed from HDFS).
> This means that the Master is unaware of Region size growth until the the next time this
chore in a RegionServer fires which is a fair amount of latency (a few minutes, by default).
Operations like flushes, compactions, and bulk-loads are delayed even though the RegionServer
is running those operations locally.
> Instead, we can create an API which these operations could invoke that would automatically
update the size of the Region being operated on. For example, a successful flush can report
that the size of a Region increased by the size of the flush. A compaction can subtract the
size of the input files of the compaction and add in the size of the resulting file.
> This de-couples the computation of a Region's size from sending the Region sizes to the
Master, allowing us to send reports more frequently, increasing the responsiveness of the
cluster to size changes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message