hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Qin Gao <q...@cs.cmu.edu>
Subject Re: Making sure the tmp directory is cleaned?
Date Mon, 22 Jun 2009 20:24:57 GMT
Thanks!

But what if the jobs get killed or failed? Does hadoop try to clean it? we
are considering bad situations - if job gets killed, will the tmp dirs sit
on local disks forever and eats up all the diskspace?

I guess this should be considered in distributed cache, but those files are
read-only, and our program will generate new temporary files.


--Q


On Mon, Jun 22, 2009 at 4:19 PM, Pankil Doshi <forpankil@gmail.com> wrote:

> Yes, If your job gets completed successfully .possibly it removes after
> completion of both map and reduce tasks.
>
> Pankil
>
> On Mon, Jun 22, 2009 at 3:15 PM, Qin Gao <qing@cs.cmu.edu> wrote:
>
> > Hi All,
> >
> > Do you know if the tmp directory on every map/reduce task will be deleted
> > automatically after the map task finishes or will do I have to delete
> them?
> >
> > I mean the tmp directory that automatically created by on current
> > directory.
> >
> > Thanks a lot
> > --Q
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message