hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11754) Make FsServerDefaults cache configurable.
Date Wed, 29 Nov 2017 20:56:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271525#comment-16271525
] 

Rushabh S Shah commented on HDFS-11754:
---------------------------------------

bq. Can you please verify that the test failures are not related.
Nevermind I ran all the failed tests locally and all of them succeeded except {{TestReadStripedFileWithMissingBlocks}}.
Failure of {{TestReadStripedFileWithMissingBlocks}}  is tracked by HDFS-12723.
{noformat}
[INFO] Running org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.154 s - in org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
[INFO] Running org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.121 s - in org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
[INFO] Running org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 83.275 s - in org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
[INFO] Running org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.54 s - in org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
[INFO] Running org.apache.hadoop.hdfs.server.federation.metrics.TestFederationMetrics
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.166 s - in org.apache.hadoop.hdfs.server.federation.metrics.TestFederationMetrics
[INFO] Running org.apache.hadoop.hdfs.TestLeaseRecovery2
[INFO] Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 69.896 s - in org.apache.hadoop.hdfs.TestLeaseRecovery2
[INFO] Running org.apache.hadoop.hdfs.TestPread
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 123.237 s - in org.apache.hadoop.hdfs.TestPread
[INFO] Running org.apache.hadoop.hdfs.TestReadStripedFileWithMissingBlocks
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 149.108 s <<<
FAILURE! - in org.apache.hadoop.hdfs.TestReadStripedFileWithMissingBlocks
[ERROR] testReadFileWithMissingBlocks(org.apache.hadoop.hdfs.TestReadStripedFileWithMissingBlocks)
 Time elapsed: 148.948 s  <<< ERROR!
java.util.concurrent.TimeoutException: Timed out waiting for /foo to have all the internalBlocks
	at org.apache.hadoop.hdfs.StripedFileTestUtil.waitBlockGroupsReported(StripedFileTestUtil.java:295)
	at org.apache.hadoop.hdfs.StripedFileTestUtil.waitBlockGroupsReported(StripedFileTestUtil.java:256)
	at org.apache.hadoop.hdfs.TestReadStripedFileWithMissingBlocks.readFileWithMissingBlocks(TestReadStripedFileWithMissingBlocks.java:98)
	at org.apache.hadoop.hdfs.TestReadStripedFileWithMissingBlocks.testReadFileWithMissingBlocks(TestReadStripedFileWithMissingBlocks.java:82)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Errors: 
[ERROR]   TestReadStripedFileWithMissingBlocks.testReadFileWithMissingBlocks:82->readFileWithMissingBlocks:98
ยป Timeout
[ERROR] Tests run: 48, Failures: 0, Errors: 1, Skipped: 0
{noformat}


Also the patch applies to branch-2.8 also. So we should be good to get in till branch-2.8.

> Make FsServerDefaults cache configurable.
> -----------------------------------------
>
>                 Key: HDFS-11754
>                 URL: https://issues.apache.org/jira/browse/HDFS-11754
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Rushabh S Shah
>            Assignee: Mikhail Erofeev
>            Priority: Minor
>              Labels: newbie
>         Attachments: HDFS-11754.001.patch, HDFS-11754.002.patch, HDFS-11754.003.patch,
HDFS-11754.004.patch, HDFS-11754.005.patch, HDFS-11754.006.patch
>
>
> DFSClient caches the result of FsServerDefaults for 60 minutes.
> But the 60 minutes time is not configurable.
> Continuing the discussion from HDFS-11702, it would be nice if we can make this configurable
and make the default as 60 minutes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message