hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <markkerz...@gmail.com>
Subject Re: part-00000.deflate as output
Date Fri, 27 Nov 2009 19:25:51 GMT
Thank you, guys, for your very useful answers

Mark

On Fri, Nov 27, 2009 at 12:44 PM, Aaron Kimball <aaron@cloudera.com> wrote:

> You are always free to run with compression disabled. But in many
> production
> situations, space or performance concerns dictate that all data sets are
> stored compressed, so I think Tim was assuming that you might be operating
> in such an environment -- in which case, you'd only need things to appear
> in
> plaintext if a human operator is inspecting the output for debugging.
>
> - Aaron
>
> On Thu, Nov 26, 2009 at 4:59 PM, Mark Kerzner <markkerzner@gmail.com>
> wrote:
>
> > It worked!
> >
> > But why is it "for testing?" I only have one job, so I need by related as
> > text, can I use this fix all the time?
> >
> > Thank you,
> > Mark
> >
> > On Thu, Nov 26, 2009 at 1:10 AM, Tim Kiefer <tim-kiefer@gmx.de> wrote:
> >
> > > For testing purposes you can also try to disable the compression:
> > >
> > > conf.setBoolean("mapred.output.compress", false);
> > >
> > > Then you can look at the output.
> > >
> > > - tim
> > >
> > >
> > > Amogh Vasekar wrote:
> > >
> > >> Hi,
> > >> ".deflate" is the default compression codec used when parameter to
> > >> generate compressed output is true ( mapred.output.compress ).
> > >> You may set the codec to be used via mapred.output.compression.codec,
> > some
> > >> commonly used are available in hadoop.io.compress package...
> > >>
> > >> Amogh
> > >>
> > >>
> > >> On 11/26/09 11:03 AM, "Mark Kerzner" <markkerzner@gmail.com> wrote:
> > >>
> > >> Hi,
> > >>
> > >> I get this part-00000.deflate instead of part-00000.
> > >>
> > >> How do I get rid of the deflate option?
> > >>
> > >> Thank you,
> > >> Mark
> > >>
> > >>
> > >>
> > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message