hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: output files written by reducers
Date Mon, 09 Jan 2012 15:29:57 GMT
Aliyeh,

I do not fully understand your question. Why even write out files you do not want?

Like any other regular FS, HDFS does not automatically clean up your old files nor has a mechanism
that sets its expiry date. Your processes or administrators should be able to build something
over HDFS that cleans up 'older' files per your needs.

On 09-Jan-2012, at 6:58 PM, aliyeh saeedi wrote:

> 
> 
> Hi
> 
> 
> Are the files written by reducers discarded? If no, after a while we have a lot of files
which are not useful and occupy disk space unduly. Is there any special parameter which can
be set to save or discard these files?  
> 
> 
> Regards :-)
> 
> 
> 
> 


Mime
View raw message