hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vishalsant <Vishal.Santo...@gmail.com>
Subject Re: Files reducing to zero byte size
Date Tue, 20 Jul 2010 23:23:34 GMT

I think I got it ..
The reason is that I do do

public void configure(JobConf job) {
		mos = new MultipleOutputs(job);
	}

for multipe output 

but I forgot ( kicking myself very very very very very hard )

/**
	 * close {@link #mos}
	 */
	public void close() throws IOException {
		if (mos != null) {
			mos.close();
		}
	}

and that causes the exception below and it does not write it.


I do see it for a time and that is pbly b'coz . the namenode feeds it from
the temp directory.
Please correct if I am wrong .



vishalsant wrote:
> 
> I ran a Map Reduce job , and created using Multiple Outputs and reduced to
> a  bunch of files
> I catted them , did a copytoLocal and all the good stuff one does.
> 
> I come back after couple of days and those files are 0 bytes. 
> 
> drwxr-xr-x   - hadoop supergroup          0 2010-07-19 14:29
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/_logs
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00000
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00001
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00002
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00003
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00004
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00005
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00006
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00007
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00008
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00009
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00010
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00011
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00012
> -rw-r--r--   3 hadoop supergroup          0 2010-07-19 14:30
> /user/hadoop/aggregations/adevent/event/2010_07_18/dba4e0ea936911dfbf590050c2490048/dd-r-00013
> 
> 
> And this has happened more than once.
> 
> 
> I am not sure where to start looking. I run fsck -blocks and it seems to
> be OK.
> 
>       Total size:    2661524498815 B (Total open files size: 60545982 B)
>        Total dirs:    7273
>        Total files:   598942 (Files currently being written: 108)
>        Total blocks (validated):      460457 (avg. block size 5780180 B)
> (Total open file blocks (not validated): 108)
>        Minimally replicated blocks:   460457 (100.0 %)
>        Over-replicated blocks:        0 (0.0 %)
>        Under-replicated blocks:       28 (0.006080915 %)
>        Mis-replicated blocks:  0 (0.0 %)
>        Default replication factor:    2
>        Average block replication:     2.947261
>        Corrupt blocks:                0
>        Missing replicas:            56 (0.0041264854 %)
>        Number of data-nodes:    8
>        Number of racks:              1
> 
> 
> I have checked the name node logs and nothing , apart from 
> 
>   
> 
> 2010-07-15 10:06:02,336 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 3 on 8020, call
> complete(/user/hadoop/aggregations/adevent/event/20100714/_temporary/_attempt_201007031521_0340_r_000031_0/hh-r-00031,
> DFSClient_attempt_201007031521_0340_r_000031_0) from 10.17.28.210:50225:
> error: java.io.IOException: Could not complete write to file
> /user/hadoop/aggregations/adevent/event/20100714/_temporary/_attempt_201007031521_0340_r_000031_0/hh-r-00031
> by DFSClient_attempt_201007031521_0340_r_000031_0
> java.io.IOException: Could not complete write to file
> /user/hadoop/aggregations/adevent/event/20100714/_temporary/_attempt_201007031521_0340_r_000031_0/hh-r-00031
> by DFSClient_attempt_201007031521_0340_r_000031_0
> 2010-07-15 10:07:10,947 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit:
> ugi=hadoop,hadoop       ip=/10.17.28.218       cmd=open      
> src=/user/hadoop/aggregations/adevent/event/20100714/hh-r-00031       
> dst=null        perm=null
> 2010-07-15 10:11:47,609 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.audit:
> ugi=hadoop,hadoop       ip=/10.36.67.130       cmd=open      
> src=/user/hadoop/aggregations/adevent/event/20100714/hh-r-00031       
> dst=null        perm=null
> 2010-07-15 11:05:51,833 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Recovering
> lease=[Lease.  Holder: DFSClient_attempt_201007031521_0340_r_000031_0,
> pendingcreates: 3],
> src=/user/hadoop/aggregations/adevent/event/20100714/hh-r-00031
> 2010-07-15 12:05:52,906 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Recovering
> lease=[Lease.  Holder: DFSClient_attempt_201007031521_0340_r_000031_0,
> pendingcreates: 3],
> src=/user/hadoop/aggregations/adevent/event/20100714/hh-r-00031
> 2010-07-15 12:05:54,607 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
> commitBlockSynchronization(newblock=blk_4668499616665973908_2986137,
> file=/user/hadoop/aggregations/adevent/event/20100714/hh-r-00031,
> newgenerationstamp=0, newlength=0, newtargets=[]) successful
> 
> 
> As I perceive this .. it did succeed ...
> 
> 
> So not sure what is happening.
> 
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/Files-reducing-to-zero-byte-size-tp29220487p29220586.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.


Mime
View raw message