avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yosi Botzer <yosi.bot...@gmail.com>
Subject bad avro file for mapreduce output
Date Mon, 06 Apr 2015 09:27:34 GMT
Hi,

I have written a map reduce job which suppose to output avro file:

job.setInputFormatClass(AvroKeyInputFormat.class);
job.setMapperClass(UserScoreMapper.class);
AvroJob.setInputKeySchema(job, GameSession.getClassSchema());
job.setMapOutputKeyClass(LongWritable.class);
AvroJob.setMapOutputValueSchema(job, UserKpi.SCHEMA$);

job.setOutputFormatClass(AvroKeyOutputFormat.class);
 job.setReducerClass(UserScoreReducer.class);
AvroJob.setOutputKeySchema(job, UserKpi.SCHEMA$);


However when I am trying to read the job output file using avro-tools I am
getting the following message:

yosi@yosi:/tmp/aaa$  java -jar ./avro-tools-1.7.7.jar tojson ./part-r-00000
Exception in thread "main" java.io.IOException: *Not an Avro data file*
at org.apache.avro.file.DataFileReader.openReader(DataFileReader.java:63)
at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:71)
at org.apache.avro.tool.Main.run(Main.java:84)
at org.apache.avro.tool.Main.main(Main.java:73)


How exactly should I setup the mapreduce job in order to produce a valid
avro file?


Yosi

Mime
View raw message