avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Binary output in MR job
Date Sat, 16 Aug 2014 14:22:13 GMT
You're not explicitly specifying a relevant form of AvroOutputFormat.
This causes the default TextOutputFormat to get used, giving you JSON
representation of records.

On Sat, Aug 16, 2014 at 3:07 PM, Anand Nalya <anand.nalya@gmail.com> wrote:
> Hi,
>
> I'm writing a MR 2 job in which I'm reading plain text as input and
> producing avro output. On running the job in local mode, the output is being
> serialized into json format. What can I do so that the output uses binary
> encoding. Following is my job definition:
>
>         Job job = new Job(getConf(), "Post convertor");
>         job.setJarByClass(getClass());
>
>         AvroJob.setOutputKeySchema(job, Post.getClassSchema());
>         AvroJob.setMapOutputKeySchema(job, Schema.create(Schema.Type.LONG));
>         AvroJob.setMapOutputValueSchema(job, Post.getClassSchema());
>
>         FileInputFormat.addInputPath(job, new Path(args[0]));
>         FileOutputFormat.setOutputPath(job, new Path(args[1]));
>
>         job.setMapperClass(PostMapper.class);
>         job.setReducerClass(PostReducer.class);
>
> Regards.
> Anand



-- 
Harsh J

Mime
View raw message