avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Software Dev <static.void....@gmail.com>
Subject Re: Schema inheritance
Date Fri, 21 Feb 2014 00:19:34 GMT
We have a similar use case and I would like to just "flatten" out the
schema by including the fields from the parent into the child without
nesting. The reason I don't want to nest is because it doesnt play well
with some of our tools (Impala, Pig, etc)

Now back to your initial response. I started playing around with the IDL
but I still can't seem to figure out how to inherit all the fields from a
parent into a child record.

I would like the final "Chile" record to look like:

{
    "type": "record",
    "name": "Child",
    "fields" : [
        {
            "name": "bar",
            "type": "string"
        },
        {
            "name": "foo",
            "type": "string"
        }
    ]
}


On Thu, Feb 20, 2014 at 12:55 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> Going back to your initial question... why don't you wish to include "foo"
> within nested "parent"?
> I am not quite getting it here.
> In my case, the nested records were of substantial size including dozens
> of fields, which then had nested records. It did not scale to write out
> AVSC definitions for the data model.
>
>
> On Thu, Feb 20, 2014 at 8:28 PM, Software Dev <static.void.dev@gmail.com>wrote:
>
>> Thanks for the input. I'm guessing then the above problem can only solved
>> with IDL and not AVSC?
>>
>>
>> On Thu, Feb 20, 2014 at 11:45 AM, Lewis John Mcgibbney <
>> lewis.mcgibbney@gmail.com> wrote:
>>
>>> Hey,
>>> Did you check out the IDL documentation?
>>> http://avro.apache.org/docs/current/idl.html
>>> I had similar data modeling issues a while back and this helped out A
>>> LOT.
>>> hth
>>>
>>>
>>> On Thu, Feb 20, 2014 at 6:58 PM, Software Dev <static.void.dev@gmail.com
>>> > wrote:
>>>
>>>> Is there anyway to include the fields of another schema into our schema
>>>> WITHOUT it creating a nested record?
>>>>
>>>>
>>>> {
>>>>     "type": "record",
>>>>     "name": "Parent",
>>>>     "fields" : [
>>>>         {
>>>>             "name": "foo",
>>>>             "type": "string"
>>>>         }
>>>>     ]
>>>> }
>>>>
>>>> {
>>>>     "type": "record",
>>>>     "name": "Child",
>>>>     "fields" : [
>>>>         {
>>>>             "name": "bar",
>>>>             "type": "string"
>>>>         },
>>>> // I dont want it nested like this
>>>> //        {
>>>> //            "name": "parent",
>>>> //            "type": "Parent"
>>>> //        }
>>>>     ]
>>>> }
>>>>
>>>> So in this example is there a way to have child include both the "bar"
>>>> field as well as "foo" without it nested under parent?
>>>>
>>>> Thanks
>>>>
>>>>
>>>
>>>
>>> --
>>> *Lewis*
>>>
>>
>>
>
>
> --
> *Lewis*
>

Mime
View raw message