hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chia-Ping Tsai (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-19321) ZKAsyncRegistry ctor would hang when zookeeper cluster is not available
Date Wed, 22 Nov 2017 16:42:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16262884#comment-16262884
] 

Chia-Ping Tsai edited comment on HBASE-19321 at 11/22/17 4:41 PM:
------------------------------------------------------------------

{code}
2017-11-22 13:13:28,358 ERROR [MemStoreFlusher.1] regionserver.HRegion(1210): Asked to modify
this region's (TestAcidGuarantees,,1511356384562.67f5ede11bc9d37151794455003c8ee9.) memstoreSize
to a negative value which is incorrect. Current memstoreSize=251900, delta=-281670
java.lang.Exception
	at org.apache.hadoop.hbase.regionserver.HRegion.checkNegativeMemStoreDataSize(HRegion.java:1210)
	at org.apache.hadoop.hbase.regionserver.HRegion.decrMemStoreSize(HRegion.java:1203)
	at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2639)
	at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2355)
	at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2327)
	at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2218)
	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:498)
	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:467)
	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:69)
	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:252)
	at java.lang.Thread.run(Thread.java:748)
{code}
It seems the eager policy cause the negative size of memstore, so it breaks the close of table
(If the memstore size is negative, the rs will be aborted when closing the region.). We don't
close the table before HBASE-19311 hence the issue doesn't hurt the {{TestAcidGuarantees}}
previously. As [~Apache9] optimized the {{TestAcidGuarantees}} by creating the mini cluster
once and creating/deleting the table for each test case, the issue hurts {{TestAcidGuarantees}}
now. I revert the HBASE-19311 locally, and then run the {{TestAcidGuarantees}} with eager
policy. The error about the negative size sill happens. We should file a issue to disable
the eager policy for {{TestAcidGuarantees}} before we have time to dig in. [~tedyu] [~Apache9]
WDYT?


was (Author: chia7712):
{code}
2017-11-22 13:13:28,358 ERROR [MemStoreFlusher.1] regionserver.HRegion(1210): Asked to modify
this region's (TestAcidGuarantees,,1511356384562.67f5ede11bc9d37151794455003c8ee9.) memstoreSize
to a negative value which is incorrect. Current memstoreSize=251900, delta=-281670
java.lang.Exception
	at org.apache.hadoop.hbase.regionserver.HRegion.checkNegativeMemStoreDataSize(HRegion.java:1210)
	at org.apache.hadoop.hbase.regionserver.HRegion.decrMemStoreSize(HRegion.java:1203)
	at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2639)
	at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2355)
	at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2327)
	at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2218)
	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:498)
	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:467)
	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:69)
	at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:252)
	at java.lang.Thread.run(Thread.java:748)
{code}
It seems the eager policy cause the negative size of memstore, so it breaks the close of table
(If the memstore size is negative, the rs will be aborted when closing the region.). We don't
close the table before HBASE-19311 hence the issue doesn't hurt the {{TestAcidGuarantees}}
previously. As [~Duo Zhang] optimized the {{TestAcidGuarantees}} by creating the mini cluster
once and creating/deleting the table for each test case, the issue hurts {{TestAcidGuarantees}}
now. I revert the HBASE-19311 locally, and then run the {{TestAcidGuarantees}} with eager
policy. The error about the negative size sill happens. We should file a issue to disable
the eager policy for {{TestAcidGuarantees}} before we have time to dig in. [~tedyu] [~Apache9]
WDYT?

> ZKAsyncRegistry ctor would hang when zookeeper cluster is not available
> -----------------------------------------------------------------------
>
>                 Key: HBASE-19321
>                 URL: https://issues.apache.org/jira/browse/HBASE-19321
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Ted Yu
>            Assignee: Guoquan Wu
>             Fix For: 2.0.0-beta-1
>
>         Attachments: HBASE-19321.master.001.patch
>
>
> From https://builds.apache.org/job/HBASE-Flaky-Tests/23477/testReport/junit/org.apache.hadoop.hbase.client/TestAdmin2/testCheckHBaseAvailableWithoutCluster/
:
> {code}
> org.junit.runners.model.TestTimedOutException: test timed out after 300000 milliseconds
> 	at org.apache.hadoop.hbase.client.TestAdmin2.testCheckHBaseAvailableWithoutCluster(TestAdmin2.java:573)
> {code}
> It seems this started hanging after HBASE-19313



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message