avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Serge Blazhievsky <easyv...@gmail.com>
Subject Re: AvroMapper map method not getting invoked
Date Thu, 29 Mar 2012 20:42:31 GMT
Can you post the code for your main method?

Serge

On Thu, Mar 29, 2012 at 12:54 PM, Deepak Nettem <deepaknettem@gmail.com>wrote:

> HI,
>
> I am trying to run a simple Avro MapReduce Mapper only job, that reads
> from an Avro source. This piece of code simply reads my avro source data
> and writes it to the outpath in the exact same schema, irrespective of what
> is given in the map method of the mapper. The map method is not getting
> invoked at all.
>
> Why is this happening?
>
> -----------------
> public class AvroReader {
>
>     //public static final Schema schema;
>     public static final Schema IN_SCHEMA =
> ReflectData.get().getSchema(Record.class);
>
>   public void AvroReaderRunner(String inputPath, String outputPath) throws
> Exception {
>
>    JobConf conf = new JobConf();
>
>    Job job = new Job(conf,"AvroRecord Job..");
>     job.setJobName("avroreader");
>     job.setJarByClass(AvroReader.class);
>
>     AvroJob.setInputSchema(conf, IN_SCHEMA);
>
>     //Schema stringSchema = Schema.create(Schema.Type.STRING);
>     //Schema pairSchema = Pair.getPairSchema(stringSchema, stringSchema);
>
>     AvroJob.setMapOutputSchema(conf, new Pair<Utf8,Utf8>(new Utf8(""),
> 0L).getSchema() );
>
>     //AvroJob.setOutputSchema(conf, IN_SCHEMA.)
>     AvroJob.setMapperClass(conf, AvroReaderMapper.class);
>
>     FileInputFormat.addInputPath(job, new Path(inputPath));
>
>     //FileInputFormat.addInputPath(job, new Path(inputPath));
>     FileOutputFormat.setOutputPath(job, new Path(outputPath));
>
>     //FileOutputFormat.setOutputPath(job, new Path(outputPath));
>
>     job.setNumReduceTasks(0);                     // map-only
>
>     job.waitForCompletion(true);
>     }
>
>   public static class AvroReaderMapper extends AvroMapper<Record,
> Pair<Utf8, Utf8>>{
>
>       @Override
>     public void map(Record in, AvroCollector<Pair<Utf8,Utf8>> collector,
>                       Reporter reporter) throws IOException {
>         collector.collect(new Pair<Utf8,Utf8>('a', 'b'));
>
>     }
>
>   }
>
> }
>
>
> --
> Warm Regards,
> Deepak Nettem
>
>
>

Mime
View raw message