hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <lars.geo...@gmail.com>
Subject Borked Splitlog
Date Mon, 16 May 2011 05:03:07 GMT
Hi,

I am on trunk and testing in pseudo distributed setup. I loaded the
machine with YCSB and got it to break at a few million inserts during
the load phase with the GC taking too long and the compaction queue
going through the roof subsequently. Since then I cannot recover the
local "cluster". It is stuck printing this:

...
2011-05-16 06:59:05,389 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
/hbase/splitlog/RESCAN0000148501 ver = 0
2011-05-16 06:59:06,388 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1
unassigned = 1
2011-05-16 06:59:06,389 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: resubmitting
unassigned task(s) after timeout
2011-05-16 06:59:06,390 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
/hbase/splitlog/RESCAN0000148502 ver = 0
2011-05-16 06:59:07,388 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1
unassigned = 1
2011-05-16 06:59:07,388 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: resubmitting
unassigned task(s) after timeout
2011-05-16 06:59:07,389 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
/hbase/splitlog/RESCAN0000148503 ver = 0
2011-05-16 06:59:08,388 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1
unassigned = 1
2011-05-16 06:59:08,388 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: resubmitting
unassigned task(s) after timeout
2011-05-16 06:59:08,389 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
/hbase/splitlog/RESCAN0000148504 ver = 0
2011-05-16 06:59:09,388 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: total tasks = 1
unassigned = 1
2011-05-16 06:59:09,389 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: resubmitting
unassigned task(s) after timeout
2011-05-16 06:59:09,390 DEBUG
org.apache.hadoop.hbase.master.SplitLogManager: task not yet acquired
/hbase/splitlog/RESCAN0000148505 ver = 0
...

This keeps on going up and up. What is the right way to recover from
this? Delete something from ZK? Delete something from HDFS? What shell
commands would help?

Thanks,
Lars

Mime
View raw message