hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Bhattacharjee <rahul.rec....@gmail.com>
Subject Re: Hadoop's Avro dependencies.
Date Wed, 22 Aug 2012 06:12:26 GMT
Well , thanks a lot Harsh. I though avro was result of hadoop's
serialization needs.

If avro isn't used for serializing maps outputs and transfer it to other
reducers then whats used for this , if not avro.

Thanks,
Rahul

On Wed, Aug 22, 2012 at 11:22 AM, Harsh J <harsh@cloudera.com> wrote:

> Hi,
>
> Hadoop doesn't use Avro serialization on its own. However, Hadoop 2.x
> does provide an AvroSerialization class you can use optionally to
> serialize using Avro libraries, and the 2.x distribution does ship an
> Avro jar along with it.
>
> On Wed, Aug 22, 2012 at 11:09 AM, Rahul Bhattacharjee
> <rahul.rec.dgp@gmail.com> wrote:
> > Hi,
> >
> > I was going through the Apache Hadoop's distribution dependencies (jars
> in
> > lib folder) and I could not find avro-1.x.x.jar.
> >
> > I though hadoop internally uses avro as its serialization mechanism for
> > intermediate data transmission (transporting maps output to reducers etc
> ),
> > so hadoop distribution must have avro within it. But it doesn't !
> >
> > Can someone enlighten me on this?
> >
> > Thanks,
> > Rahul
> >
>
>
>
> --
> Harsh J
>

Mime
View raw message