hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Job-wide cleanup functionality?
Date Thu, 03 Feb 2011 16:39:12 GMT
Perhaps this has been covered before, but I wasn't able to dig up any info.

Is there any way to run a custom "job cleanup" for a map/reduce job?  I 
know that each map and reduce has a cleanup method, which can be used to 
clean up at the end of each task.  But what I want is to run a single 
cleanup step that runs after all the reducers complete.  Is there any 
way to do this in M/R?  I didn't see one.  Would be nice to have an API 
like this to work with:

job.setCleanupClass(MyCleanupClass.class)

Also, if no such functionality exists, what's my next best workaround to 
achieve this on a M/R job?

Thanks,

DR

Mime
View raw message