hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15446) WASB: PageBlobInputStream.skip breaks HBASE replication
Date Mon, 07 May 2018 10:59:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465783#comment-16465783
] 

Steve Loughran commented on HADOOP-15446:
-----------------------------------------

+1, committed to branch 3.1 & trunk. If you want backporting to branch-2, run the tests,
tell me how it went, & I'll backport.

I did see one failure in my own test run, I'm assuming unrelated and just a function of network
distance.
{code}
[ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 26.213 s <<<
FAILURE! - in org.apache.hadoop.fs.azure.TestClientThrottlingAnalyzer
[ERROR] testManySuccessAndErrorsAndWaiting(org.apache.hadoop.fs.azure.TestClientThrottlingAnalyzer)
 Time elapsed: 1.123 s  <<< FAILURE!
java.lang.AssertionError: The actual value 9 is not within the expected range: [5.60, 8.40].
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.hadoop.fs.azure.TestClientThrottlingAnalyzer.fuzzyValidate(TestClientThrottlingAnalyzer.java:46)
	at org.apache.hadoop.fs.azure.TestClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting(TestClientThrottlingAnalyzer.java:168)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}


> WASB: PageBlobInputStream.skip breaks HBASE replication
> -------------------------------------------------------
>
>                 Key: HADOOP-15446
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15446
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/azure
>    Affects Versions: 2.9.0, 3.0.2
>            Reporter: Thomas Marquardt
>            Assignee: Thomas Marquardt
>            Priority: Major
>         Attachments: HADOOP-15446-001.patch, HADOOP-15446-002.patch, HADOOP-15446-003.patch
>
>
> Page Blobs are primarily used by HBASE.  HBASE replication, which apparently has not
been used with WASB until recently, performs non-sequential reads on log files using PageBlobInputStream. 
There are bugs in this stream implementation which prevent skip and seek from working properly, and
eventually the stream state becomes corrupt and unusable.
> I believe this bug affects all releases of WASB/HADOOP.  It appears to be a day-0 bug
in PageBlobInputStream.  There were similar bugs opened in the past (HADOOP-15042) but the
issue was not properly fixed, and no test coverage was added.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message