hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saptarshi Guha <saptarshi.g...@gmail.com>
Subject 60gb set of files copy - warning?
Date Tue, 27 May 2008 15:22:24 GMT
Hello,
I have a folder with 391 subfolders, each of which contains 1000  
files. Mostly in 100's of kb but some few in 100'smb and a few in 10's  
of GB.
While copying( all files got copied i think) i got the following  
messages (replication=2)

Do you think some files weren't fully copied?
Should i check file sizes of each and compare with my local copy?
Thanks
Saptarshi


  $HADOOPBIN dfs -put all /network  <-----started at 6.00 in evening
08/05/26 22:28:38 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Bad connect ack with  
firstBadLink 128.210.196.96:8910
08/05/26 22:28:38 INFO dfs.DFSClient: Abandoning block  
blk_-2796307691713955505_234661
08/05/26 22:28:44 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Bad connect ack with  
firstBadLink 128.210.141.106:8910
08/05/26 22:28:44 INFO dfs.DFSClient: Abandoning block  
blk_8793679776152772897_234662
08/05/26 22:28:50 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Bad connect ack with  
firstBadLink 128.210.141.107:8910
08/05/26 22:28:50 INFO dfs.DFSClient: Abandoning block  
blk_4359282278588771942_234662
08/05/26 22:28:56 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Could not read from stream
08/05/26 22:28:56 INFO dfs.DFSClient: Abandoning block  
blk_1344112447969705949_234662
08/05/26 22:29:03 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Bad connect ack with  
firstBadLink 128.210.196.95:8910
08/05/26 22:29:03 INFO dfs.DFSClient: Abandoning block  
blk_-7828475613961395136_234663
08/05/26 22:29:09 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Bad connect ack with  
firstBadLink 128.210.196.98:8910
08/05/26 22:29:09 INFO dfs.DFSClient: Abandoning block  
blk_6272663323502917304_234663
08/05/26 22:29:15 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Bad connect ack with  
firstBadLink 128.210.196.95:8910
08/05/26 22:29:15 INFO dfs.DFSClient: Abandoning block  
blk_5555319980529981632_234663
08/05/26 22:29:21 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Bad connect ack with  
firstBadLink 128.210.141.107:8910
08/05/26 22:29:21 INFO dfs.DFSClient: Abandoning block  
blk_6983796963942228061_234663
08/05/26 22:29:27 WARN dfs.DFSClient: DataStreamer Exception:  
java.io.IOException: Unable to create new block.
08/05/26 22:29:27 WARN dfs.DFSClient: Error Recovery for block  
blk_6983796963942228061_234663 bad datanode[1]
08/05/26 22:29:27 WARN dfs.DFSClient: DFSOutputStream  
ResponseProcessor exception  for block  
blk_-429558087382368087_234664java.io.IOException: Bad response 1 for  
block bl
k_-429558087382368087_234664 from datanode 128.210.141.106:8910
         at org.apache.hadoop.dfs.DFSClient$DFSOutputStream 
$ResponseProcessor.run(DFSClient.java:2017)

08/05/26 22:29:27 WARN dfs.DFSClient: Error Recovery for block  
blk_-429558087382368087_234664 bad datanode[1] 128.210.141.106:8910
08/05/26 22:29:27 WARN dfs.DFSClient: Error Recovery for block  
blk_-429558087382368087_234664 in pipeline 128.210.196.109:8910,  
128.210.141.106:8910: bad datanode 128.210.
141.106:8910
08/05/26 22:29:27 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Could not read from stream
08/05/26 22:29:27 INFO dfs.DFSClient: Abandoning block  
blk_-6280724692326442086_234665
08/05/26 22:29:33 INFO dfs.DFSClient: Exception in  
createBlockOutputStream java.io.IOException: Could not read from stream
08/05/26 22:29:33 INFO dfs.DFSClient: Abandoning block  
blk_-2918561222217292919_234665


Mime
View raw message