hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-733) TestBlockReport fails intermittently
Date Wed, 04 Nov 2009 02:05:32 GMT

    [ https://issues.apache.org/jira/browse/HDFS-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773347#action_12773347
] 

Konstantin Shvachko commented on HDFS-733:
------------------------------------------

Looks like Hudson did not change his opinion on this.
Looking at the log. Seems that replication has actually already happened by the time the tests
starts waiting for the temporary replica.

{code}
2009-11-03 23:15:26,213 INFO  hdfs.StateChange (BlockManager.java:computeReplicationWorkForBlock(857))
- BLOCK* ask 127.0.0.1:60191 to replicate blk_3759837087059880694_1001 to datanode(s) 127.0.0.1:47781
2009-11-03 23:15:26,427 INFO  datanode.DataNode (DataNode.java:transferBlock(1072)) - DatanodeRegistration(127.0.0.1:60191,
storageID=DS-681753498-127.0.1.1-60191-1257290123417, infoPort=54109, ipcPort=41992) Starting
thread to transfer block blk_3759837087059880694_1001 to 127.0.0.1:47781 
2009-11-03 23:15:26,431 INFO  datanode.DataNode (DataXceiver.java:opWriteBlock(224)) - Receiving
block blk_3759837087059880694_1001 src: /127.0.0.1:38882 dest: /127.0.0.1:47781
2009-11-03 23:15:26,450 INFO  datanode.DataNode (DataNode.java:run(1265)) - DatanodeRegistration(127.0.0.1:60191,
storageID=DS-681753498-127.0.1.1-60191-1257290123417, infoPort=54109, ipcPort=41992):Transmitted
block blk_3759837087059880694_1001 to /127.0.0.1:47781
2009-11-03 23:15:26,458 DEBUG datanode.DataNode (BlockReceiver.java:receivePacket(443)) -
Receiving one packet for block blk_3759837087059880694_1001 of length 0 seqno 3 offsetInBlock
3072000 lastPacketInBlock true
2009-11-03 23:15:26,458 DEBUG datanode.DataNode (BlockReceiver.java:receivePacket(476)) -
Receiving an empty packet or the end of the block blk_3759837087059880694_1001
2009-11-03 23:15:26,458 DEBUG datanode.DataNode (FSDataset.java:addBlock(136)) - addBlock:
Moved /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/build/test/data/dfs/data/data3/tmp/blk_3759837087059880694_1001.meta
to /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/build/test/data/dfs/data/data3/current/finalized/blk_3759837087059880694_1001.meta
2009-11-03 23:15:26,459 DEBUG datanode.DataNode (FSDataset.java:addBlock(137)) - addBlock:
Moved /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/build/test/data/dfs/data/data3/tmp/blk_3759837087059880694
to /grid/0/hudson/hudson-slave/workspace/Hdfs-Patch-h5.grid.sp2.yahoo.net/trunk/build/test/data/dfs/data/data3/current/finalized/blk_3759837087059880694
2009-11-03 23:15:26,459 INFO  datanode.DataNode (DataXceiver.java:opWriteBlock(369)) - Received
block blk_3759837087059880694_1001 src: /127.0.0.1:38882 dest: /127.0.0.1:47781 of size 3072000

2009-11-03 23:15:26,461 DEBUG hdfs.StateChange (FSNamesystem.java:blockReceived(2848)) - BLOCK*
NameSystem.blockReceived: blk_3759837087059880694_1001 is received from 127.0.0.1:47781
2009-11-03 23:15:26,461 DEBUG namenode.FSNamesystem (PendingReplicationBlocks.java:remove(87))
- Removing pending replication for blockblk_3759837087059880694_1001
2009-11-03 23:15:26,461 DEBUG namenode.FSNamesystem (DatanodeDescriptor.java:processReportedBlock(466))
- Reported block blk_3759837087059880694_1001 on 127.0.0.1:47781 size 3072000 replicaState
= FINALIZED
2009-11-03 23:15:26,462 INFO  hdfs.StateChange (BlockManager.java:addStoredBlock(1102)) -
BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:47781 is added to blk_3759837087059880694_1001
size 3072000
{code}

And then you start waiting in the test:
{code}
2009-11-03 23:15:26,471 DEBUG datanode.TestBlockReport (TestBlockReport.java:prepareSecondReplica(600))
- Replica state before the loop 0
2009-11-03 23:15:26,571 DEBUG datanode.TestBlockReport (TestBlockReport.java:prepareSecondReplica(605))
- Keep waiting for blk_3759837087059880694 is in state 0
2009-11-03 23:15:26,671 DEBUG datanode.TestBlockReport (TestBlockReport.java:prepareSecondReplica(605))
- Keep waiting for blk_3759837087059880694 is in state 0
2009-11-03 23:15:26,772 DEBUG datanode.TestBlockReport (TestBlockReport.java:prepareSecondReplica(605))
- Keep waiting for blk_3759837087059880694 is in state 0
2009-11-03 23:15:26,872 DEBUG datanode.TestBlockReport (TestBlockReport.java:prepareSecondReplica(605))
- Keep waiting for blk_3759837087059880694 is in state 0
{code}

So waiting longer does not help.

> TestBlockReport fails intermittently
> ------------------------------------
>
>                 Key: HDFS-733
>                 URL: https://issues.apache.org/jira/browse/HDFS-733
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.21.0
>            Reporter: Suresh Srinivas
>            Assignee: Konstantin Boudnik
>             Fix For: 0.21.0, 0.22.0
>
>         Attachments: HDFS-733.patch
>
>
> Details at http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/58/testReport/

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message