hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AnilKumar B <akumarb2...@gmail.com>
Subject Re: Multiple inputs for different avro inputs
Date Thu, 27 Feb 2014 17:20:28 GMT
Hi Yong,

One question is that why you setInputKeySchema? Does your job load the Avro
data as the key to the following Mapper?
>> Yes, I am loading data as a key.

Thanks & Regards,
B Anil Kumar.


On Thu, Feb 27, 2014 at 7:57 PM, java8964 <java8964@hotmail.com> wrote:

> Using the Union schema is correct, which should be able to support multi
> schemas input.
>
> One question is that why you setInputKeySchema? Does your job load the
> Avro data as the key to the following Mapper?
>
> Yong
>
> ------------------------------
> Date: Thu, 27 Feb 2014 16:13:34 +0530
> Subject: Multiple inputs for different avro inputs
> From: akumarb2010@gmail.com
> To: user@hadoop.apache.org
>
>
> Hi,
>
> I am using MultipleInputs to read two different avro inputs with different
> schemas.
>
> But in run method, as we need to specify
> the AvroJob.setInputKeySchema(job,schema),
>
> Which schema I need to set?
>
> I tried as below
>
>             List<Schema> schemas = new ArrayList<Schema>();
> schemas.add(FlumeEvent.SCHEMA$);
>  schemas.add(Event.SCHEMA$);
> AvroJob.setInputKeySchema(testJob, Schema.createUnion(schemas));
>
> But I am facing issue while Map phase
> *Error: org.apache.avro.AvroTypeException: Found Event, expecting union*
>
> How to fix this issue?
>
> *One more doubt: Why we don't have AvroMultipleInputs just like
> AvroMultipleOutputs? Any reason?*
>
> Thanks & Regards,
> B Anil Kumar.
>

Mime
View raw message