hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Kunz <ck...@yahoo-inc.com>
Subject CommitBlockSynchronization issue
Date Fri, 04 Sep 2009 19:06:54 GMT
Using hadoop-0.18.3,
One of our files with size 273592 changed size to an incorrect value 262144
due to commitBlockSynchronization.

Would that have been fixed by HADOOP-5134?

Thanks,
-Christian

Here are the relevant log messages of the namenode

log.2009-09-01:2009-09-01 20:52:45,451 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.allocateBlock:
fileName.blk_2101888387690035515_63606967

log.2009-09-01:2009-09-01 20:57:43,535 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: xxx.yyy.zzz.234:54610 is added
to blk_2101888387690035515_63606967 size 273592

log.2009-09-01:2009-09-01 20:57:47,847 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: xxx.yyy.zzz.250:54610 is added
to blk_2101888387690035515_63606967 size 273592

log.2009-09-01:2009-09-01 20:58:01,480 INFO
org.apache.hadoop.fs.FSNamesystem:
commitBlockSynchronization(lastblock=blk_2101888387690035515_63606967,
newgenerationstamp=63637360, newlength=262144,
newtargets=[xxx.yyy.zzz.234:54610])

log.2009-09-01:2009-09-01 20:58:01,482 INFO
org.apache.hadoop.fs.FSNamesystem:
commitBlockSynchronization(blk_2101888387690035515_63637360) successful

log.2009-09-01:2009-09-01 21:01:07,508 INFO
org.apache.hadoop.fs.FSNamesystem:
commitBlockSynchronization(lastblock=blk_2101888387690035515_63637360,
newgenerationstamp=63639267, newlength=262144,
newtargets=[xxx.yyy.zzz.234:54610])

log.2009-09-01:2009-09-01 21:01:07,512 INFO
org.apache.hadoop.fs.FSNamesystem:
commitBlockSynchronization(blk_2101888387690035515_63639267) successful

log.2009-09-01:2009-09-01 21:01:08,946 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
ask xxx.yyy.zzz.234:54610 to replicate blk_2101888387690035515_63639267 to
datanode(s) xxx.yyy.zzz.230:54610 xxx.yyy.zzz.115:54610

log.2009-09-01:2009-09-01 21:01:12,061 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: xxx.yyy.zzz.230:54610 is added
to blk_2101888387690035515_63639267 size 262144

log.2009-09-01:2009-09-01 21:01:12,097 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.addStoredBlock: blockMap updated: xxx.yyy.zzz.115:54610 is added
to blk_2101888387690035515_63639267 size 262144

log.2009-09-01:2009-09-01 21:35:03,362 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
NameSystem.processReport: block blk_2101888387690035515_63637360 on
xxx.yyy.zzz.250:54610 size 273592 does not belong to any file.

log.2009-09-01:2009-09-01 21:35:06,344 INFO
org.apache.hadoop.dfs.StateChange: BLOCK*
ask xxx.yyy.zzz.250:54610 to delete blk_2101888387690035515_63637360



Mime
View raw message