hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From java8964 <java8...@hotmail.com>
Subject RE: Multiple inputs for different avro inputs
Date Thu, 27 Feb 2014 14:27:15 GMT
Using the Union schema is correct, which should be able to support multi schemas input.
One question is that why you setInputKeySchema? Does your job load the Avro data as the key
to the following Mapper?

Date: Thu, 27 Feb 2014 16:13:34 +0530
Subject: Multiple inputs for different avro inputs
From: akumarb2010@gmail.com
To: user@hadoop.apache.org

I am using MultipleInputs to read two different avro inputs with different schemas.
But in run method, as we need to specify the AvroJob.setInputKeySchema(job,schema),

Which schema I need to set?
I tried as below 
            List<Schema> schemas = new ArrayList<Schema>();		schemas.add(FlumeEvent.SCHEMA$);
		schemas.add(Event.SCHEMA$);		AvroJob.setInputKeySchema(testJob, Schema.createUnion(schemas));

But I am facing issue while Map phaseError: org.apache.avro.AvroTypeException: Found Event,
expecting union

How to fix this issue?
One more doubt: Why we don't have AvroMultipleInputs just like AvroMultipleOutputs? Any reason?

Thanks & Regards,
B Anil Kumar.

View raw message