hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pankil Doshi <forpan...@gmail.com>
Subject Re: Making sure the tmp directory is cleaned?
Date Mon, 22 Jun 2009 20:34:51 GMT
No..If your job gets killed or failed.Temp wont clean up.. and In that case
you will have to carefully clean that on your own. If you dont clean it up
yourself it will eat up your disk space.

Pankil

On Mon, Jun 22, 2009 at 4:24 PM, Qin Gao <qing@cs.cmu.edu> wrote:

> Thanks!
>
> But what if the jobs get killed or failed? Does hadoop try to clean it? we
> are considering bad situations - if job gets killed, will the tmp dirs sit
> on local disks forever and eats up all the diskspace?
>
> I guess this should be considered in distributed cache, but those files are
> read-only, and our program will generate new temporary files.
>
>
> --Q
>
>
> On Mon, Jun 22, 2009 at 4:19 PM, Pankil Doshi <forpankil@gmail.com> wrote:
>
> > Yes, If your job gets completed successfully .possibly it removes after
> > completion of both map and reduce tasks.
> >
> > Pankil
> >
> > On Mon, Jun 22, 2009 at 3:15 PM, Qin Gao <qing@cs.cmu.edu> wrote:
> >
> > > Hi All,
> > >
> > > Do you know if the tmp directory on every map/reduce task will be
> deleted
> > > automatically after the map task finishes or will do I have to delete
> > them?
> > >
> > > I mean the tmp directory that automatically created by on current
> > > directory.
> > >
> > > Thanks a lot
> > > --Q
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message