hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhangduo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13065) Increasing -Xmx when running TestDistributedLogSplitting
Date Sat, 21 Feb 2015 05:56:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14330016#comment-14330016
] 

zhangduo commented on HBASE-13065:
----------------------------------

[~stack] Seems HBase-TRUNK jenkins is running on a 32bit machine? So there are only 4GB address
space. We increase heap from 1900m to 2800m, then there is much less address space for native
memory.
In TestAcidGuarantees, we create connection per read write thread but do not close it, so
we create lots of AsyncRpcClient instance. Each instance has its own netty thread pool, so
there are too many threads and will easily exceed the 4GB address space(VIRT, not RES) limit
and then make the test fail.

{noformat}
java.lang.OutOfMemoryError: unable to create new native thread
	at java.lang.Thread.start0(Native Method)
	at java.lang.Thread.start(Thread.java:693)
	at io.netty.util.concurrent.SingleThreadEventExecutor.shutdownGracefully(SingleThreadEventExecutor.java:557)
	at io.netty.util.concurrent.MultithreadEventExecutorGroup.shutdownGracefully(MultithreadEventExecutorGroup.java:146)
	at io.netty.util.concurrent.AbstractEventExecutorGroup.shutdownGracefully(AbstractEventExecutorGroup.java:69)
	at org.apache.hadoop.hbase.ipc.AsyncRpcClient.close(AsyncRpcClient.java:253)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.internalClose(ConnectionManager.java:2373)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.close(ConnectionManager.java:2386)
	at org.apache.hadoop.hbase.HBaseTestingUtility.shutdownMiniCluster(HBaseTestingUtility.java:1055)
	at org.apache.hadoop.hbase.TestAcidGuarantees.testGetAtomicity(TestAcidGuarantees.java:346)
{noformat}

I wonder why we run jenkins job on 32bit machines? Seems PreCommit-Build is run on 64bit machines.

> Increasing -Xmx when running TestDistributedLogSplitting
> --------------------------------------------------------
>
>                 Key: HBASE-13065
>                 URL: https://issues.apache.org/jira/browse/HBASE-13065
>             Project: HBase
>          Issue Type: Bug
>          Components: test
>            Reporter: zhangduo
>            Assignee: zhangduo
>             Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
>
>         Attachments: 13065-fix.txt
>
>
> Found this in PreCommit Build reports
> https://builds.apache.org/job/PreCommit-HBASE-Build/12885/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestDistributedLogSplitting-output.txt
> {noformat}
> 2015-02-18 03:45:42,141 WARN  [RS:4;asf901:41265] util.Sleeper(97): We slept 59018ms
instead of 1000ms, this is likely due to a long garbage collecting pause and it's usually
bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
> 2015-02-18 03:45:26,750 WARN  [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(167): Detected
pause in JVM or host machine (eg GC): pause of approximately 39767ms
> GC pool 'PS MarkSweep' had collection(s): count=65 time=47720ms
> {noformat}
> Maybe we should increase the max heap size since this test starts 6 regionservers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message