avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anand Nalya <anand.na...@gmail.com>
Subject Re: Binary output in MR job
Date Sat, 16 Aug 2014 14:41:03 GMT
Thanks Harsh, It worked after specifying AvroKeyOutputFormat as output
format.


On 16 August 2014 19:52, Harsh J <harsh@cloudera.com> wrote:

> You're not explicitly specifying a relevant form of AvroOutputFormat.
> This causes the default TextOutputFormat to get used, giving you JSON
> representation of records.
>
> On Sat, Aug 16, 2014 at 3:07 PM, Anand Nalya <anand.nalya@gmail.com>
> wrote:
> > Hi,
> >
> > I'm writing a MR 2 job in which I'm reading plain text as input and
> > producing avro output. On running the job in local mode, the output is
> being
> > serialized into json format. What can I do so that the output uses binary
> > encoding. Following is my job definition:
> >
> >         Job job = new Job(getConf(), "Post convertor");
> >         job.setJarByClass(getClass());
> >
> >         AvroJob.setOutputKeySchema(job, Post.getClassSchema());
> >         AvroJob.setMapOutputKeySchema(job,
> Schema.create(Schema.Type.LONG));
> >         AvroJob.setMapOutputValueSchema(job, Post.getClassSchema());
> >
> >         FileInputFormat.addInputPath(job, new Path(args[0]));
> >         FileOutputFormat.setOutputPath(job, new Path(args[1]));
> >
> >         job.setMapperClass(PostMapper.class);
> >         job.setReducerClass(PostReducer.class);
> >
> > Regards.
> > Anand
>
>
>
> --
> Harsh J
>

Mime
View raw message