avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shirahatti, Nikhil" <snik...@telenav.com>
Subject Re: Avro Map Reduce Question: GenericRecord, renaming reduce output
Date Tue, 12 Jun 2012 17:24:03 GMT
That¹s right. The junit test, did not do any asserts on the file checking.
I've checked it in, so please try again. However, if you try to open the
file in /logshed you'll probably see what I'm talking about.

I also tried setting AvroJob before job instantiation, but I got the same
error.

Snippet:
JobConf jobConf = new
JobConf(LogshedCollectorUtils.getLocalHadoopConfiguartion());
	
		AvroJob.setInputSchema(jobConf, IN_SCHEMA);
		AvroJob.setOutputSchema(jobConf, OUT_SCHEMA);

		AvroJob.setMapperClass(jobConf, LogshedMapper.class);
		AvroJob.setReducerClass(jobConf, LogshedReducer.class);
		
		Job job = new Job(jobConf, "muxdemux_job");

		FileInputFormat.setInputPaths(job, new Path(args[0]));
		Path outPath = new Path(args[1]);
		FileOutputFormat.setOutputPath(job, outPath);
		job.setJarByClass(MuxDemuxJob.class);



Thanks,
Nikhil


On 6/12/12 10:05 AM, "Doug Cutting" <cutting@apache.org> wrote:

>When I do 'git clone https://github.com/snikhil0/avro-mr.git; cd
>avro-mr; ant test', I see:
>
>    [junit] Running
>com.telenav.logshed.collector.muxdemux.MuxDemuxRunnableTest
>    [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 12.637 sec
>
>BUILD SUCCESSFUL
>
>Finally, Nikhil suggested above that your problem is in
>MuxDemuxJob.java, where you set properties on the JobConf after
>creating the Job.  The AvroJob methods should instead be called before
>the Job is constructed.
>
>Doug


Mime
View raw message