hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10000) Initiate lease recovery for outstanding WAL files at the very beginning of recovery
Date Mon, 09 Dec 2013 19:40:09 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843459#comment-13843459
] 

Ted Yu commented on HBASE-10000:
--------------------------------

I tried to add a test which simulates the absence of isFileClosed method using following code:
{code}
    Mockito.when(dfs.isFileClosed(FILE)).thenThrow(NoSuchMethodException.class);
{code}
where dfs is mocked off DistributedFileSystem.
However, the NoSuchMethodException is only thrown at time of invocation - reflection discovers
isFileClosed method:
{code}
2013-12-09 11:29:21,090 WARN  [Thread-2] util.FSHDFSUtils(218): Failed invocation for /Users/tyu/trunk/hbase-server/target/test-data/55827920-583b-4f05-8a7e-2bc3e8bf29e5/file.txt
java.lang.reflect.InvocationTargetException
	at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.hbase.util.FSHDFSUtils.isFileClosed(FSHDFSUtils.java:214)
	at org.apache.hadoop.hbase.util.FSHDFSUtils.recoverDFSFileLease(FSHDFSUtils.java:138)
	at org.apache.hadoop.hbase.util.TestFSHDFSUtils.testIsFileClosedAbsent(TestFSHDFSUtils.java:117)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
Caused by: java.lang.NoSuchMethodException
	... 15 more
{code}

> Initiate lease recovery for outstanding WAL files at the very beginning of recovery
> -----------------------------------------------------------------------------------
>
>                 Key: HBASE-10000
>                 URL: https://issues.apache.org/jira/browse/HBASE-10000
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>             Fix For: 0.98.1
>
>         Attachments: 10000-0.96-v5.txt, 10000-0.96-v6.txt, 10000-recover-ts-with-pb-2.txt,
10000-recover-ts-with-pb-3.txt, 10000-recover-ts-with-pb-4.txt, 10000-recover-ts-with-pb-5.txt,
10000-recover-ts-with-pb-6.txt, 10000-v4.txt, 10000-v5.txt, 10000-v6.txt
>
>
> At the beginning of recovery, master can send lease recovery requests concurrently for
outstanding WAL files using a thread pool.
> Each split worker would first check whether the WAL file it processes is closed.
> Thanks to Nicolas Liochon and Jeffery discussion with whom gave rise to this idea. 



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message