avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debu Silverman <silverd...@gmail.com>
Subject Re: AvroMultipleOutputs writing empty filed
Date Wed, 29 Apr 2015 00:05:57 GMT
Debu Silverman <silverdebu@...> writes:

> 
> 
> Trying to write output into two different named output file using 
AvroMultipleOutputs but getting an empty file and no error in the logs. It 
works when writing to a single file.
> 
> Avro version 1.7.1
> 
> Thanks
> Debu
> 


Code for the job setup....

=======================

Job job = new Job(config, "AVRO_MULTITEST");
job.setJarByClass(AvroMultiWriter.class);

FileInputFormat.setInputPaths(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));

job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(AvroKeyValueOutputFormat.class);
job.setMapperClass(AvroMultiWriteMapper.class);
job.setNumReduceTasks(0);


AvroJob.setOutputKeySchema(job, Schema.create(Schema.Type.STRING));
AvroJob.setOutputValueSchema(job, schema);


AvroMultipleOutputs.setCountersEnabled(job, true);
AvroMultipleOutputs.addNamedOutput(job,"F1",
		AvroKeyValueOutputFormat.class, Schema.create
(Schema.Type.STRING),schema);

AvroMultipleOutputs.addNamedOutput(job,"F2",
		AvroKeyValueOutputFormat.class, Schema.create
(Schema.Type.STRING),schema);

LazyOutputFormat.setOutputFormatClass(job, AvroKeyValueOutputFormat.class);

===============================================================

Thanks,
Debu


Mime
View raw message