hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Esteban Gutierrez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12369) Warn if hbase.bucketcache.size too close or equal to MaxDirectMemorySize
Date Wed, 29 Oct 2014 00:22:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187780#comment-14187780
] 

Esteban Gutierrez commented on HBASE-12369:
-------------------------------------------

That sounds reasonable. Something like: 

FATAL Bucket cache size (5G) cannot be equal to MaxDirectMemorySize (5GB). Reduce hbase.bucketcache.size
or increase MaxDirectMemorySize in your configuration.

and a warning if too close:

WARN  Bucket cache size (4.75G) is 95% of MaxDirectMemorySize (5GB). Reduce hbase.bucketcache.size
or increase MaxDirectMemorySize in your configuration. See bucket cache section in the reference
guide for further information.

 



> Warn if hbase.bucketcache.size too close or equal to MaxDirectMemorySize
> ------------------------------------------------------------------------
>
>                 Key: HBASE-12369
>                 URL: https://issues.apache.org/jira/browse/HBASE-12369
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>            Reporter: Esteban Gutierrez
>
> Our ref guide currently says that its required to leave some room from the DirectMemory.
However if hbase.bucketcache.size is too close or equal to MaxDirectMemorySize it can trigger
OOMEs: 
> {code}
> 2014-10-28 16:14:41,585 INFO  [master//172.16.0.101:16020] util.ByteBufferArray: Allocating
buffers total=5.00 GB, sizePerBuffer=4 MB, count=1280, direct=true
> 2014-10-28 16:14:41,604 INFO  [172.16.0.101:16020.activeMasterManager] master.ServerManager:
Waiting for region servers count to settle; currently checked in 1, slept for 99 ms, expecting
minimum of 2, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms.
> 2014-10-28 16:14:43,144 INFO  [172.16.0.101:16020.activeMasterManager] master.ServerManager:
Waiting for region servers count to settle; currently checked in 1, slept for 1639 ms, expecting
minimum of 2, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms.
> 2014-10-28 16:14:44,057 INFO  [master//172.16.0.101:16020] regionserver.HRegionServer:
STOPPED: Failed initialization
> 2014-10-28 16:14:44,058 ERROR [master//172.16.0.101:16020] regionserver.HRegionServer:
Failed init
> java.lang.OutOfMemoryError: Direct buffer memory
> 	at java.nio.Bits.reserveMemory(Bits.java:658)
> 	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
> 	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
> 	at org.apache.hadoop.hbase.util.ByteBufferArray.<init>(ByteBufferArray.java:65)
> 	at org.apache.hadoop.hbase.io.hfile.bucket.ByteBufferIOEngine.<init>(ByteBufferIOEngine.java:47)
> 	at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getIOEngineFromName(BucketCache.java:310)
> 	at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.<init>(BucketCache.java:218)
> 	at org.apache.hadoop.hbase.io.hfile.CacheConfig.getL2(CacheConfig.java:513)
> 	at org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(CacheConfig.java:536)
> 	at org.apache.hadoop.hbase.io.hfile.CacheConfig.<init>(CacheConfig.java:213)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1259)
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:818)
> 	at java.lang.Thread.run(Thread.java:724)
> {code}
> It would be helpful to print a warn message that hbase.bucketcache.size too close or
equal to MaxDirectMemorySize.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message