Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id F394A200CC2 for ; Wed, 21 Jun 2017 00:28:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id F2296160BEF; Tue, 20 Jun 2017 22:28:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1DBDF160BE1 for ; Wed, 21 Jun 2017 00:28:05 +0200 (CEST) Received: (qmail 43923 invoked by uid 500); 20 Jun 2017 22:28:05 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 43912 invoked by uid 99); 20 Jun 2017 22:28:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Jun 2017 22:28:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id C528ACCF8B for ; Tue, 20 Jun 2017 22:28:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.01 X-Spam-Level: X-Spam-Status: No, score=-100.01 tagged_above=-999 required=6.31 tests=[SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id VQyQSzTc2mds for ; Tue, 20 Jun 2017 22:28:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 829F35F523 for ; Tue, 20 Jun 2017 22:28:02 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 08076E0D4D for ; Tue, 20 Jun 2017 22:28:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 34364240C5 for ; Tue, 20 Jun 2017 22:28:00 +0000 (UTC) Date: Tue, 20 Jun 2017 22:28:00 +0000 (UTC) From: "Hadoop QA (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-18133) Low-latency space quota size reports MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 20 Jun 2017 22:28:07 -0000 [ https://issues.apache.org/jira/browse/HBASE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16056581#comment-16056581 ] Hadoop QA commented on HBASE-18133: ----------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 43s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} | {color:green} Patch does not have any anti-patterns. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 12 new or modified test files. {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 31s {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m 17s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s {color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 47s {color} | {color:green} master passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 8s {color} | {color:red} hbase-server in master has 12 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 7s {color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 17s {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 15s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 6s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 6s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 36s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green} 33m 34s {color} | {color:green} Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 28s {color} | {color:red} hbase-server generated 1 new + 12 unchanged - 0 fixed = 13 total (was 12) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s {color} | {color:green} hbase-hadoop-compat in the patch passed. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s {color} | {color:green} hbase-hadoop2-compat generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s {color} | {color:green} hbase-server in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 21s {color} | {color:green} hbase-hadoop-compat in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 26s {color} | {color:green} hbase-hadoop2-compat in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 22s {color} | {color:red} hbase-server in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 32s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 79m 8s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hbase-server | | | Nullcheck of HRegion.rsServices at line 2764 of value previously dereferenced in org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(WAL, MonitoredTask, HRegion$PrepareFlushResult, Collection) At HRegion.java:2764 of value previously dereferenced in org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(WAL, MonitoredTask, HRegion$PrepareFlushResult, Collection) At HRegion.java:[line 2682] | | Failed junit tests | hadoop.hbase.regionserver.TestMinVersions | | | hadoop.hbase.filter.TestFilterFromRegionSide | | | hadoop.hbase.io.hfile.TestScannerSelectionUsingKeyRange | | | hadoop.hbase.regionserver.TestKeepDeletes | | | hadoop.hbase.io.encoding.TestPrefixTree | | | hadoop.hbase.coprocessor.TestCoprocessorInterface | | | hadoop.hbase.regionserver.TestScanWithBloomError | | | hadoop.hbase.client.TestIntraRowPagination | | | hadoop.hbase.regionserver.TestBlocksScanned | | | hadoop.hbase.filter.TestMultipleColumnPrefixFilter | | | hadoop.hbase.regionserver.TestStoreFileRefresherChore | | | hadoop.hbase.regionserver.TestScanner | | | hadoop.hbase.regionserver.TestWALMonotonicallyIncreasingSeqId | | | hadoop.hbase.filter.TestInvocationRecordFilter | | | hadoop.hbase.filter.TestFilter | | | hadoop.hbase.filter.TestDependentColumnFilter | | | hadoop.hbase.regionserver.TestResettingCounters | | | hadoop.hbase.coprocessor.TestRegionObserverStacking | | | hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan | | | hadoop.hbase.regionserver.TestWideScanner | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12873737/HBASE-18133.002.patch | | JIRA Issue | HBASE-18133 | | Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux aff9a79f8183 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh | | git revision | master / 5b485d1 | | Default Java | 1.8.0_131 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html | | findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/artifact/patchprocess/new-findbugs-hbase-server.html | | unit | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/artifact/patchprocess/patch-unit-hbase-server.txt | | unit test logs | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/artifact/patchprocess/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/testReport/ | | modules | C: hbase-hadoop-compat hbase-hadoop2-compat hbase-server U: . | | Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/7258/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > Low-latency space quota size reports > ------------------------------------ > > Key: HBASE-18133 > URL: https://issues.apache.org/jira/browse/HBASE-18133 > Project: HBase > Issue Type: Improvement > Reporter: Josh Elser > Assignee: Josh Elser > Fix For: 3.0.0 > > Attachments: HBASE-18133.001.patch, HBASE-18133.002.patch > > > Presently space quota enforcement relies on RegionServers sending reports to the master about each Region that they host. This is done by periodically, reading the cached size of each HFile in each Region (which was ultimately computed from HDFS). > This means that the Master is unaware of Region size growth until the the next time this chore in a RegionServer fires which is a fair amount of latency (a few minutes, by default). Operations like flushes, compactions, and bulk-loads are delayed even though the RegionServer is running those operations locally. > Instead, we can create an API which these operations could invoke that would automatically update the size of the Region being operated on. For example, a successful flush can report that the size of a Region increased by the size of the flush. A compaction can subtract the size of the input files of the compaction and add in the size of the resulting file. > This de-couples the computation of a Region's size from sending the Region sizes to the Master, allowing us to send reports more frequently, increasing the responsiveness of the cluster to size changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)