incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: Invalid schema created in AvrosTest#testNestedTables
Date Tue, 03 Jul 2012 14:35:11 GMT
On Tue, Jul 3, 2012 at 4:26 PM, Josh Wills <josh.wills@gmail.com> wrote:
> On Tue, Jul 3, 2012 at 12:42 AM, Gabriel Reid <gabriel.reid@gmail.com> wrote:
>>
>> While implementing the map side joins, I needed to make the PType
>> interface extend Serializable, and that caused me to stumble upon an
>> issue in AvrosTest. The testNestedTables method creates a nested table
>> schema (unsurprisingly), with the call in question being equivalent to
>> this:
>>
>>     Avros.tableOf(Avros.strings(), Avros.tableOf(Avros.ints(),
>> Avros.doubles()));
>>
>> This results in an invalid schema being created due to the same
>> namespace and name (org.apache.avro.mapred.Pair) being used twice in
>> the schema.
...
>
> I added it as a convenience method for a project I was working on-- my
> thought was that if you tried to put a PTableType inside of another
> PTableType, we should detect it and automatically convert the nested
> PTableType to a pairs(keyType, valueType), as you indicated was the
> right thing to do. I'm surprised that it didn't work properly, since I
> thought that I was doing the conversion from PTableType to
> pairs(keyType, valueType) when the PType was being constructed. Do you
> mind if I take a look at it first and try to fix it?

Yep, please do -- if there's a use case for it, it definitely makes
sense to keep it and fix it. You can spot the issue if you call
toString on the schema that is created (in the nested table unit
test).

- Gabriel

Mime
View raw message