hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Qin Gao <q...@cs.cmu.edu>
Subject Re: Making sure the tmp directory is cleaned?
Date Mon, 22 Jun 2009 20:46:20 GMT
Thanks, then I will try keep a log on the files and clean them out, thanks.
--Q


On Mon, Jun 22, 2009 at 4:34 PM, Pankil Doshi <forpankil@gmail.com> wrote:

> No..If your job gets killed or failed.Temp wont clean up.. and In that case
> you will have to carefully clean that on your own. If you dont clean it up
> yourself it will eat up your disk space.
>
> Pankil
>
> On Mon, Jun 22, 2009 at 4:24 PM, Qin Gao <qing@cs.cmu.edu> wrote:
>
> > Thanks!
> >
> > But what if the jobs get killed or failed? Does hadoop try to clean it?
> we
> > are considering bad situations - if job gets killed, will the tmp dirs
> sit
> > on local disks forever and eats up all the diskspace?
> >
> > I guess this should be considered in distributed cache, but those files
> are
> > read-only, and our program will generate new temporary files.
> >
> >
> > --Q
> >
> >
> > On Mon, Jun 22, 2009 at 4:19 PM, Pankil Doshi <forpankil@gmail.com>
> wrote:
> >
> > > Yes, If your job gets completed successfully .possibly it removes after
> > > completion of both map and reduce tasks.
> > >
> > > Pankil
> > >
> > > On Mon, Jun 22, 2009 at 3:15 PM, Qin Gao <qing@cs.cmu.edu> wrote:
> > >
> > > > Hi All,
> > > >
> > > > Do you know if the tmp directory on every map/reduce task will be
> > deleted
> > > > automatically after the map task finishes or will do I have to delete
> > > them?
> > > >
> > > > I mean the tmp directory that automatically created by on current
> > > > directory.
> > > >
> > > > Thanks a lot
> > > > --Q
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message