avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nir_zamir <nir.za...@gmail.com>
Subject Parsing a Pair's value - inherited namespace?
Date Tue, 09 Apr 2013 16:02:30 GMT
Hi,

I noticed that after calling:

/AvroJob.setMapOutputSchema(conf,
Pair.getPairSchema(Schema.create(Type.INT), schema));/
(schema is parsed from an avro file, and has no namespace)

When the M/R job is run, there's a call to /AvroJob.getJobOutputSchema
/which calls /Schema.parse/ - which parses the schema I set in
setMapOutputSchema.

The problem: the schema (the value in the Pair returned by
getJobOutputSchema) has no namespace, so it gets it from the Pair (which is
"org.apache.avro.mapred"), so my schema's full name is
"org.apache.avro.mapred.MySchema".

Why do I care? Well, when the schema is a union, the avro items passed to
the AvroMapper.map have no namespace (so their full name is "MySchema", and
are not matching the mapper's output schema (I get a "not in union"
exception). Note that this happens although I pass a Pair to collect(), but
in this case, the Pair's namespace is ignored.

I hope it's clear - will be happy to provide any more details.

Your help is much appreciated!!

Thanks,
Nir




--
View this message in context: http://apache-avro.679487.n3.nabble.com/Parsing-a-Pair-s-value-inherited-namespace-tp4026810.html
Sent from the Avro - Users mailing list archive at Nabble.com.

Mime
View raw message