hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Bhattacharjee <rahul.rec....@gmail.com>
Subject Re: no _SUCCESS file in MR output directory.
Date Mon, 06 May 2013 07:43:48 GMT
I will shoot an email to oozie dev as well.

Thanks,
Rahul


On Mon, May 6, 2013 at 12:54 PM, Harsh J <harsh@cloudera.com> wrote:

> I don't see where Oozie does it explicitly in code, but thats better
> discussed over the Oozie's user@oozie.apache.org lists.
>
> That said, if something is implicitly behaving in a certain way, it
> may have a good reason to do so thats worth investigating before
> toggling.
>
> On Mon, May 6, 2013 at 12:34 PM, Rahul Bhattacharjee
> <rahul.rec.dgp@gmail.com> wrote:
> > Oozie is being used for triggering the MR job. Looks like oozie disables
> the
> > success file creation using the configuration that you have mentioned for
> > FileOutputCommitter.
> >
> > I have enabled it by setting this property in conf.
> >
> > Rahul
> >
> >
> > On Mon, May 6, 2013 at 9:38 AM, Rahul Bhattacharjee
> > <rahul.rec.dgp@gmail.com> wrote:
> >>
> >> Thanks Harsh for the pointers. I will find out more on this.
> >>
> >>
> >> On Sun, May 5, 2013 at 11:26 PM, Harsh J <harsh@cloudera.com> wrote:
> >>>
> >>> I can think of a few, most obvious ones:
> >>>
> >>> 1. Job didn't succeed and/or the file was deleted (*shields self*)
> >>> 2. Job overrode the default FileOutputCommitter with something that
> >>> doesn't do success marking.
> >>> 3. Job specifically asked to not create such files, via config
> >>> mapreduce.fileoutputcommitter.marksuccessfuljobs or so, set to false.
> >>>
> >>> On Sun, May 5, 2013 at 9:54 PM, Rahul Bhattacharjee
> >>> <rahul.rec.dgp@gmail.com> wrote:
> >>> > Hi,
> >>> >
> >>> >
> >>> > A few days back , I was going through a MR job's output , but there
> >>> > wasn't
> >>> > any _SUCCESS file in the output directory.
> >>> > I was wondering what all reasons for this  (no _SUCCESS file)?
> >>> >
> >>> > Thanks,
> >>> > Rahul
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>
> >>
> >
>
>
>
> --
> Harsh J
>

Mime
View raw message