hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood" <stuh...@webmail.us>
Subject Removing files after processing
Date Fri, 24 Aug 2007 13:43:49 GMT


Whats the best way to go about doing cleanup after MapReduce jobs? I'd like to have the job
delete its input files when it has finished successfully (but preferably before it is marked
as having finished: so I don't have to deal with a race condition).

Obviously, I don't want to have to track which files are being processed for each job, since
that data is stored anyway? Also, I'm using JobClient.submitJob(), so I can't sit around and
wait to do the cleanup manually.

Any suggestions?


Stu Hood
"You manage your business. We'll manage your email."®
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message