avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Software Dev <static.void....@gmail.com>
Subject Re: Schema inheritance
Date Fri, 21 Feb 2014 00:22:13 GMT
I should also note that the IDL introduced another concept that we
previously were dealing with... protocol. Is the protocol just a grouping
of related records?

FYI, we are using Avro strictly for serialization/de-serialization and no
RPC features.


On Thu, Feb 20, 2014 at 4:19 PM, Software Dev <static.void.dev@gmail.com>wrote:

> We have a similar use case and I would like to just "flatten" out the
> schema by including the fields from the parent into the child without
> nesting. The reason I don't want to nest is because it doesnt play well
> with some of our tools (Impala, Pig, etc)
>
> Now back to your initial response. I started playing around with the IDL
> but I still can't seem to figure out how to inherit all the fields from a
> parent into a child record.
>
> I would like the final "Chile" record to look like:
>
> {
>     "type": "record",
>     "name": "Child",
>     "fields" : [
>         {
>             "name": "bar",
>             "type": "string"
>         },
>         {
>             "name": "foo",
>             "type": "string"
>         }
>     ]
> }
>
>
> On Thu, Feb 20, 2014 at 12:55 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
>> Going back to your initial question... why don't you wish to include
>> "foo" within nested "parent"?
>> I am not quite getting it here.
>> In my case, the nested records were of substantial size including dozens
>> of fields, which then had nested records. It did not scale to write out
>> AVSC definitions for the data model.
>>
>>
>> On Thu, Feb 20, 2014 at 8:28 PM, Software Dev <static.void.dev@gmail.com>wrote:
>>
>>> Thanks for the input. I'm guessing then the above problem can only
>>> solved with IDL and not AVSC?
>>>
>>>
>>> On Thu, Feb 20, 2014 at 11:45 AM, Lewis John Mcgibbney <
>>> lewis.mcgibbney@gmail.com> wrote:
>>>
>>>> Hey,
>>>> Did you check out the IDL documentation?
>>>> http://avro.apache.org/docs/current/idl.html
>>>> I had similar data modeling issues a while back and this helped out A
>>>> LOT.
>>>> hth
>>>>
>>>>
>>>> On Thu, Feb 20, 2014 at 6:58 PM, Software Dev <
>>>> static.void.dev@gmail.com> wrote:
>>>>
>>>>> Is there anyway to include the fields of another schema into our
>>>>> schema WITHOUT it creating a nested record?
>>>>>
>>>>>
>>>>> {
>>>>>     "type": "record",
>>>>>     "name": "Parent",
>>>>>     "fields" : [
>>>>>         {
>>>>>             "name": "foo",
>>>>>             "type": "string"
>>>>>         }
>>>>>     ]
>>>>> }
>>>>>
>>>>> {
>>>>>     "type": "record",
>>>>>     "name": "Child",
>>>>>     "fields" : [
>>>>>         {
>>>>>             "name": "bar",
>>>>>             "type": "string"
>>>>>         },
>>>>> // I dont want it nested like this
>>>>> //        {
>>>>> //            "name": "parent",
>>>>> //            "type": "Parent"
>>>>> //        }
>>>>>     ]
>>>>> }
>>>>>
>>>>> So in this example is there a way to have child include both the "bar"
>>>>> field as well as "foo" without it nested under parent?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Lewis*
>>>>
>>>
>>>
>>
>>
>> --
>> *Lewis*
>>
>
>

Mime
View raw message