hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9744) TestDirectoryScanner#testThrottling occasionally time out after 300 seconds
Date Tue, 02 Feb 2016 21:54:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129109#comment-15129109

Daniel Templeton commented on HDFS-9744:

The problem is that there have to be enough blocks to make the scanning take long enough to
be able to measure the throttling.  Since we can't predict where the test will run, it's hard
to pin that number.  Too high makes the test take forever.  Too low and the test breaks. 
Feel free to try to tune the block count, but understand that you're kinda stuck between a
rock and a hard place.

> TestDirectoryScanner#testThrottling occasionally time out after 300 seconds
> ---------------------------------------------------------------------------
>                 Key: HDFS-9744
>                 URL: https://issues.apache.org/jira/browse/HDFS-9744
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode
>         Environment: Jenkins
>            Reporter: Wei-Chiu Chuang
>            Priority: Minor
>              Labels: test
> I have seen quite a few test failures in TestDirectoryScanner#testThrottling.
> https://builds.apache.org/job/Hadoop-Hdfs-trunk/2793/testReport/org.apache.hadoop.hdfs.server.datanode/TestDirectoryScanner/testThrottling/
> Looking at the log, it does not look like the test got stucked. On my local machine,
this test took 219 seconds. It is likely that this test takes more than 300 seconds to complete
on a busy jenkins slave. I think it is reasonable to set a longer time out value, or reduce
the number of blocks to reduce the duration of the test.
> Error Message
> {noformat}
> test timed out after 300000 milliseconds
> {noformat}
> Stacktrace
> {noformat}
> java.lang.Exception: test timed out after 300000 milliseconds
> 	at java.lang.Object.wait(Native Method)
> 	at java.lang.Object.wait(Object.java:503)
> 	at org.apache.hadoop.hdfs.DataStreamer.waitAndQueuePacket(DataStreamer.java:804)
> 	at org.apache.hadoop.hdfs.DFSOutputStream.enqueueCurrentPacket(DFSOutputStream.java:423)
> 	at org.apache.hadoop.hdfs.DFSOutputStream.enqueueCurrentPacketFull(DFSOutputStream.java:432)
> 	at org.apache.hadoop.hdfs.DFSOutputStream.writeChunk(DFSOutputStream.java:418)
> 	at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:217)
> 	at org.apache.hadoop.fs.FSOutputSummer.write1(FSOutputSummer.java:125)
> 	at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:111)
> 	at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:57)
> 	at java.io.DataOutputStream.write(DataOutputStream.java:107)
> 	at org.apache.hadoop.hdfs.DFSTestUtil.createFile(DFSTestUtil.java:418)
> 	at org.apache.hadoop.hdfs.DFSTestUtil.createFile(DFSTestUtil.java:376)
> 	at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.createFile(TestDirectoryScanner.java:108)
> 	at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:584)
> {noformat}

This message was sent by Atlassian JIRA

View raw message