avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: bad avro file for mapreduce output
Date Fri, 10 Apr 2015 07:19:18 GMT
The setup looks alright to me at first glance. What does the file's
header look like? Is it being written as a text file?

On Mon, Apr 6, 2015 at 2:57 PM, Yosi Botzer <yosi.botzer@gmail.com> wrote:
> Hi,
>
> I have written a map reduce job which suppose to output avro file:
>
> job.setInputFormatClass(AvroKeyInputFormat.class);
> job.setMapperClass(UserScoreMapper.class);
> AvroJob.setInputKeySchema(job, GameSession.getClassSchema());
> job.setMapOutputKeyClass(LongWritable.class);
> AvroJob.setMapOutputValueSchema(job, UserKpi.SCHEMA$);
>
> job.setOutputFormatClass(AvroKeyOutputFormat.class);
> job.setReducerClass(UserScoreReducer.class);
> AvroJob.setOutputKeySchema(job, UserKpi.SCHEMA$);
>
>
> However when I am trying to read the job output file using avro-tools I am
> getting the following message:
>
> yosi@yosi:/tmp/aaa$  java -jar ./avro-tools-1.7.7.jar tojson ./part-r-00000
> Exception in thread "main" java.io.IOException: Not an Avro data file
> at org.apache.avro.file.DataFileReader.openReader(DataFileReader.java:63)
> at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:71)
> at org.apache.avro.tool.Main.run(Main.java:84)
> at org.apache.avro.tool.Main.main(Main.java:73)
>
>
> How exactly should I setup the mapreduce job in order to produce a valid
> avro file?
>
>
> Yosi



-- 
Harsh J

Mime
View raw message