hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark question <markq2...@gmail.com>
Subject Re: outputCollector vs. Localfile
Date Fri, 20 May 2011 16:06:56 GMT
I thought it was, because of FileBytesWritten counter. Thanks for the
clarification.
Mark

On Fri, May 20, 2011 at 4:23 AM, Harsh J <harsh@cloudera.com> wrote:

> Mark,
>
> On Fri, May 20, 2011 at 10:17 AM, Mark question <markq2011@gmail.com>
> wrote:
> > This is puzzling me ...
> >
> >  With a mapper producing output of size ~ 400 MB ... which one is
> supposed
> > to be faster?
> >
> >  1) output collector: which will write to local file then copy to HDFS
> since
> > I don't have reducers.
>
> A regular map-only job does not write to the local FS, it writes to
> the HDFS directly (i.e., a local DN if one is found).
>
> --
> Harsh J
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message