hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharath Vissapragada <bhara...@cloudera.com>
Subject Re: Distributed log splitting failing after cluster outage.
Date Sun, 09 Mar 2014 18:33:21 GMT
Check if there are an 0 sized wals in /hbase/.logs and sideline them and
restart. That could help. As Ted mentioned the actual problematic log names
are in the RS logs that got the task assigned.


On Fri, Mar 7, 2014 at 12:43 AM, David Koch <ogdude@googlemail.com> wrote:

> Hello,
>
> Our HBase cluster had an unexpected shut-down and while trying to bring it
> back up we the Master gets stuck with the following message:
>
> Failed splitting of [ list of <host_name>,<port>,<tmst> ]
> java.io.IOException: error or interrupted while splitting logs in [ list of
> <host_name>,<port>,<tmst> ]
> Task = installed = 10 done = 0 error = 10
> at
>
> org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:282)
> at
>
> org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:300)
> at
>
> org.apache.hadoop.hbase.master.MasterFileSystem.splitLogAfterStartup(MasterFileSystem.java:242)
> at
>
> org.apache.hadoop.hbase.master.HMaster.splitLogAfterStartup(HMaster.java:661)
> at
>
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:580)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:396)
> at java.lang.Thread.run(Thread.java:724)
>
> What can I do to get the cluster operational again. There was no data
> ingestion going on since quite some hours before the crash so maybe
> clearing out /hbase/.logs/ could be an option.
>
> Thanks,
>
> /David
>



-- 
Bharath Vissapragada
<http://www.cloudera.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message