hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gustavo Anatoly (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9948) HMaster should handle duplicate log split requests
Date Wed, 08 Jan 2014 20:22:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13865848#comment-13865848
] 

Gustavo Anatoly commented on HBASE-9948:
----------------------------------------

Hi, [~jeffreyz].

I will follow your suggestions and really to avoid data loss the request splitting log process
should be an atomic operation, so the best way is investigate the root causes of dup log.

[~yuzhihong@gmail.com], How can I reproduce this scenario?

Thank you [~jeffreyz].

> HMaster should handle duplicate log split requests
> --------------------------------------------------
>
>                 Key: HBASE-9948
>                 URL: https://issues.apache.org/jira/browse/HBASE-9948
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Ted Yu
>            Assignee: Gustavo Anatoly
>         Attachments: HBASE-9948-v2.patch, HBASE-9948.patch
>
>
> I saw the following in test output for TestRestartCluster:
> {code}
> 2013-11-11 19:59:55,538 DEBUG [M:0;kiyo:36213] master.SplitLogManager(327): Scheduling
batch of logs to split
> 2013-11-11 19:59:55,538 INFO  [M:0;kiyo:36213] master.SplitLogManager(329): started splitting
1 logs in [hdfs://localhost:46376/user/hortonzy/hbase/WALs/kk,44962,1384199990193-splitting]
> 2013-11-11 19:59:55,538 WARN  [M:0;kiyo:36213] master.SplitLogManager(1048): Failure
because two threads can't wait for the same task; path=/hbase/splitWAL/WALs%2Fkk%2C44962%2C1384199990193-splitting%2Fkk%252C44962%252C1384199990193.1384199993702.meta
> 2013-11-11 19:59:55,538 FATAL [M:0;kiyo:36213] master.HMaster(2188): Master server abort:
loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
> 2013-11-11 19:59:55,538 FATAL [M:0;kiyo:36213] master.HMaster(2193): Unhandled exception.
Starting shutdown.
> java.io.IOException: duplicate log split scheduled for hdfs://localhost:46376/user/hortonzy/hbase/WALs/kk,44962,1384199990193-splitting/kk%2C44962%2C1384199990193.1384199993702.meta
>         at org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:343)
>         at org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:409)
>         at org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:301)
>         at org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:292)
>         at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:1038)
>         at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:868)
>         at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:605)
>         at java.lang.Thread.run(Thread.java:724)
> 2013-11-11 19:59:55,539 INFO  [M:0;kiyo:36213] master.HMaster(2386): Aborting
> 2013-11-11 19:59:55,539 DEBUG [M:0;kiyo:36213] master.HMaster(1234): Stopping service
threads
> {code}
> HMaster should handle duplicate log split requests, instead of aborting.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message