hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Serega Sheypak <serega.shey...@gmail.com>
Subject Re: Can't run hbase testing util cluster using CDH 5.4.4
Date Mon, 20 Jul 2015 21:27:40 GMT
I see these lines:

2015-07-20 21:27:21,791 INFO  [RegionOpenAndInitThread-hbase:namespace-1]
regionserver.HRegion (HRegion.java:createHRegion(5598)) - creating HRegion
hbase:namespace HTD == 'hbase:namespace', {NAME => 'info', BLOOMFILTER =>
'ROW', VERSIONS => '10', IN_MEMORY => 'true', KEEP_DELETED_CELLS =>
'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION =>
'NONE', CACHE_DATA_IN_L1 => 'true', MIN_VERSIONS => '0', BLOCKCACHE =>
'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} RootDir =
file:/Users/ssa/devel/myown/hadoop/mini-hdfs-cluster-maven-plugin/target/hbase-root/.tmp
Table name == hbase:namespace

2015-07-20 21:27:21,802 INFO  [RegionOpenAndInitThread-hbase:namespace-1]
regionserver.HRegion (HRegion.java:doClose(1425)) - Closed
hbase:namespace,,1437420441756.a69bbbcaf4a82786964da8d9cc62bea3.

2015-07-20 21:27:21,861 INFO  [MASTER_TABLE_OPERATIONS-SERGEYs-MBP:61781-0]
hbase.MetaTableAccessor (MetaTableAccessor.java:addRegionsToMeta(1169)) -
Added 2

2015-07-20 21:27:21,864 WARN  [MASTER_TABLE_OPERATIONS-SERGEYs-MBP:61781-0]
zookeeper.ZKTableStateManager (ZKTableStateManager.java:setTableState(100))
- Moving table hbase:namespace state from ENABLING to ENABLED

2015-07-20 21:27:21,866 INFO  [MASTER_TABLE_OPERATIONS-SERGEYs-MBP:61781-0]
handler.CreateTableHandler (CreateTableHandler.java:completed(219)) -
Table, hbase:namespace, creation successful


then it hungs and prints thread dump in few minutes.


2015-07-20 21:25 GMT+02:00 Esteban Gutierrez <esteban@cloudera.com>:

>
> But do you see that thread printing anything in the logs?
>
> --
> Cloudera, Inc.
>
>
> On Mon, Jul 20, 2015 at 12:07 PM, Serega Sheypak <serega.sheypak@gmail.com
> > wrote:
>
>> This is testing utiliy, it has few bytes of data to load. Running on
>> oracle-jdk8
>>
>> java.lang.Thread.run(Thread.java:745)
>> Thread 178 (JvmPauseMonitor):
>>   State: TIMED_WAITING
>>   Blocked count: 3
>>   Waited count: 398
>>   Stack:
>>     java.lang.Thread.sleep(Native Method)
>>
>> org.apache.hadoop.hbase.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:159)
>>     java.lang.Thread.run(Thread.java:745)
>>
>> Thread 177 (JvmPauseMonitor):
>>   State: TIMED_WAITING
>>   Blocked count: 1
>>   Waited count: 398
>>   Stack:
>>     java.lang.Thread.sleep(Native Method)
>>
>> org.apache.hadoop.hbase.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:159)
>>     java.lang.Thread.run(Thread.java:745)
>>
>> 2015-07-20 19:30 GMT+02:00 Esteban Gutierrez <esteban@cloudera.com>:
>>
>>> -user@hbase
>>>
>>> Hi Serega,
>>>
>>> The RunTimeException is pointing to a timeout of nearly 3min. Have tried
>>> to find in the master log lines what is causing that 3min pause? do you see
>>> any log line related to the JvmPauseMonitor? (perhaps some GC going on)
>>>
>>> thanks,
>>> esteban.
>>>
>>> --
>>> Cloudera, Inc.
>>>
>>>
>>> On Mon, Jul 20, 2015 at 12:28 AM, Serega Sheypak <
>>> serega.sheypak@gmail.com> wrote:
>>>
>>>> @Sean, thanks. I saw sometimes Cloudera guys help here. I also used
>>>> Cloudera community forum.
>>>>
>>>> @Jean-Marc, nothing special, just maven-plugin wrapper around
>>>> miniHbaseCluster
>>>> Here is the code where failure happen: new
>>>> MiniHBaseCluster(configuration,
>>>> 1);
>>>>
>>>> There are nothing special from my side. I'm surprised, it always worked
>>>> since CDH 4.4, just bump dependency versions, fix code to follow API
>>>> changes and that' all.
>>>>
>>>> 2015-07-20 4:08 GMT+02:00 Jean-Marc Spaggiari <jean-marc@spaggiari.org
>>>> >:
>>>>
>>>> > Hi Serega,
>>>> >
>>>> > What kind of tests are your trying to run? The HBase test suite? Or
>>>> > something you developed yourself?
>>>> >
>>>> > JM
>>>> >
>>>> > 2015-07-19 17:47 GMT-04:00 Serega Sheypak <serega.sheypak@gmail.com>:
>>>> >
>>>> > > Hi, bumped my testing stuff to CDH 5.4.4 and got failure while
>>>> running
>>>> > > tests. Here is a log
>>>> > >
>>>> > > 2015-07-19 23:40:18,607 INFO
>>>> [SERGEYs-MBP:51977.activeMasterManager]
>>>> > > master.TableNamespaceManager (TableNamespaceManager.java:start(85))
>>>> -
>>>> > > Namespace table not found. Creating...
>>>> > >
>>>> > > 2015-07-19 23:40:18,620 INFO  [ProcessThread(sid:0 cport:-1):]
>>>> > > server.PrepRequestProcessor
>>>> (PrepRequestProcessor.java:pRequest(645)) -
>>>> > Got
>>>> > > user-level KeeperException when processing
>>>> sessionid:0x14ea84290170000
>>>> > > type:create cxid:0x1ac zxid:0x35 txntype:-1 reqpath:n/a Error
>>>> > > Path:/hbase/table-lock/hbase:namespace Error:KeeperErrorCode =
>>>> NoNode for
>>>> > > /hbase/table-lock/hbase:namespace
>>>> > >
>>>> > > 2015-07-19 23:40:18,630 INFO
>>>> > [MASTER_TABLE_OPERATIONS-SERGEYs-MBP:51977-0]
>>>> > > handler.CreateTableHandler (CreateTableHandler.java:process(189))
-
>>>> > Create
>>>> > > table hbase:namespace
>>>> > >
>>>> > > 2015-07-19 23:40:18,644 INFO
>>>> [RegionOpenAndInitThread-hbase:namespace-1]
>>>> > > regionserver.HRegion (HRegion.java:createHRegion(5598)) - creating
>>>> > HRegion
>>>> > > hbase:namespace HTD == 'hbase:namespace', {NAME => 'info',
>>>> BLOOMFILTER =>
>>>> > > 'ROW', VERSIONS => '10', IN_MEMORY => 'true', KEEP_DELETED_CELLS
=>
>>>> > > 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER',
>>>> COMPRESSION =>
>>>> > > 'NONE', CACHE_DATA_IN_L1 => 'true', MIN_VERSIONS => '0',
BLOCKCACHE
>>>> =>
>>>> > > 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} RootDir
=
>>>> > >
>>>> > >
>>>> >
>>>> file:/Users/ssa/devel/myown/hadoop/mini-hdfs-cluster-maven-plugin/target/hbase-root/.tmp
>>>> > > Table name == hbase:namespace
>>>> > >
>>>> > > 2015-07-19 23:40:18,655 INFO
>>>> [RegionOpenAndInitThread-hbase:namespace-1]
>>>> > > regionserver.HRegion (HRegion.java:doClose(1425)) - Closed
>>>> > > hbase:namespace,,1437342018607.9c2e23572f970747f86ec499b89c281b.
>>>> > >
>>>> > > 2015-07-19 23:40:18,702 INFO
>>>> > [MASTER_TABLE_OPERATIONS-SERGEYs-MBP:51977-0]
>>>> > > hbase.MetaTableAccessor
>>>> (MetaTableAccessor.java:addRegionsToMeta(1169)) -
>>>> > > Added 2
>>>> > >
>>>> > > 2015-07-19 23:40:18,704 WARN
>>>> > [MASTER_TABLE_OPERATIONS-SERGEYs-MBP:51977-0]
>>>> > > zookeeper.ZKTableStateManager
>>>> > (ZKTableStateManager.java:setTableState(100))
>>>> > > - Moving table hbase:namespace state from ENABLING to ENABLED
>>>> > >
>>>> > > 2015-07-19 23:40:18,706 INFO
>>>> > [MASTER_TABLE_OPERATIONS-SERGEYs-MBP:51977-0]
>>>> > > handler.CreateTableHandler (CreateTableHandler.java:completed(219))
>>>> -
>>>> > > Table, hbase:namespace, creation successful
>>>> > >
>>>> > > Process Thread Dump: Thread dump because: Master not initialized
>>>> after
>>>> > > 200000ms seconds
>>>> > >
>>>> > > 186 active threads
>>>> > >
>>>> > > Thread 255 (MASTER_TABLE_OPERATIONS-SERGEYs-MBP:51977-0):
>>>> > >
>>>> > >   State: WAITING
>>>> > >
>>>> > >   Blocked count: 11
>>>> > >
>>>> > >   Waited count: 11
>>>> > >
>>>> > >   Waiting on
>>>> > >
>>>> > >
>>>> >
>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@20b5f2ac
>>>> > >
>>>> > >   Stack:
>>>> > >
>>>> > >     sun.misc.Unsafe.park(Native Method)
>>>> > >
>>>> > >
>>>>  java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>>>> > >
>>>> > >
>>>> > >
>>>> > >
>>>> >
>>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>>>> > >
>>>> > >
>>>> > >
>>>> >
>>>> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
>>>> > >
>>>> > >
>>>> > >
>>>> > >
>>>> >
>>>> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
>>>> > >
>>>> > >
>>>> > >
>>>> > >
>>>> >
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
>>>> > >
>>>> > >
>>>> > >
>>>> > >
>>>> >
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>> > >
>>>> > >     java.lang.Thread.run(Thread.java:745)
>>>> > >
>>>> > > Thread 249 (CatalogJanitor-SERGEYs-MBP:51977):
>>>> > >
>>>> > >
>>>> > > ====
>>>> > >
>>>> > >
>>>> > > 2015-07-19 23:43:33,188 ERROR [main] hbase.MiniHBaseCluster
>>>> > > (MiniHBaseCluster.java:init(229)) - Error starting cluster
>>>> > >
>>>> > > java.lang.RuntimeException: Master not initialized after 200000ms
>>>> seconds
>>>> > >
>>>> > > at
>>>> > >
>>>> > >
>>>> >
>>>> org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:225)
>>>> > >
>>>> > > at
>>>> > >
>>>> > >
>>>> >
>>>> org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:436)
>>>> > >
>>>> > > at
>>>> >
>>>> org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:224)
>>>> > >
>>>> > > at
>>>> > >
>>>> org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:93)
>>>> > >
>>>> > > at
>>>> > >
>>>> org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:80)
>>>> > >
>>>> > >  at
>>>> > >
>>>> org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:67)
>>>> > >
>>>> > >
>>>> > > fails here: MINI_HBASE_CLUSTER = new
>>>> MiniHBaseCluster(configuration, 1);
>>>> > >
>>>> > >
>>>> > > What it could be? Have no idea.
>>>> > >
>>>> >
>>>>
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message