Mailing-List: contact user-help@avro.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@avro.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.219.44 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAH-BiJgEU3eHvZJTm_M=4h551kfAv6265c9_dqqk5u-F8iGs0A@mail.gmail.com>
References: 
 <CAH-BiJhzhy+1TE09AJe2zkXbg_PJk8qmrEOM=coSLvzVNc671Q@mail.gmail.com>
 <CAGaRif1nff_KvvVO=GL2gukK2SuDeX6Neass7A-PWcScRtYPTw@mail.gmail.com>
 <CAH-BiJhU5+GYr+PP8oO+iX35xCWYng_sck9Eq8WHy_Yvi3ZV0A@mail.gmail.com>
 <CAGaRif2YidTGoAGitM-tkTs9uWwJb1ssi4bnxFZ0Ri8wzTHw4w@mail.gmail.com>
 <CAH-BiJg2CgPiNEr5K8x8sVTTm_m38wzhUAiiE4EO_sRecMaL3A@mail.gmail.com>
 <CAH-BiJgEU3eHvZJTm_M=4h551kfAv6265c9_dqqk5u-F8iGs0A@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Fri, 21 Feb 2014 17:32:52 +0530
Message-ID: 
 <CAOcnVr08tTUg27jn=CyekZz360eR4PBhtiTfTT3njTs=cgJwcQ@mail.gmail.com>
Subject: Re: Schema inheritance
To: "user@avro apache. org" <user@avro.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Use of protocol schemas in IDLs is optional.

On Fri, Feb 21, 2014 at 5:52 AM, Software Dev <static.void.dev@gmail.com> wrote:
> I should also note that the IDL introduced another concept that we
> previously were dealing with... protocol. Is the protocol just a grouping of
> related records?
>
> FYI, we are using Avro strictly for serialization/de-serialization and no
> RPC features.
>
>
> On Thu, Feb 20, 2014 at 4:19 PM, Software Dev <static.void.dev@gmail.com>
> wrote:
>>
>> We have a similar use case and I would like to just "flatten" out the
>> schema by including the fields from the parent into the child without
>> nesting. The reason I don't want to nest is because it doesnt play well with
>> some of our tools (Impala, Pig, etc)
>>
>> Now back to your initial response. I started playing around with the IDL
>> but I still can't seem to figure out how to inherit all the fields from a
>> parent into a child record.
>>
>> I would like the final "Chile" record to look like:
>>
>> {
>>     "type": "record",
>>     "name": "Child",
>>     "fields" : [
>>         {
>>             "name": "bar",
>>             "type": "string"
>>         },
>>         {
>>             "name": "foo",
>>             "type": "string"
>>         }
>>     ]
>> }
>>
>>
>> On Thu, Feb 20, 2014 at 12:55 PM, Lewis John Mcgibbney
>> <lewis.mcgibbney@gmail.com> wrote:
>>>
>>> Going back to your initial question... why don't you wish to include
>>> "foo" within nested "parent"?
>>> I am not quite getting it here.
>>> In my case, the nested records were of substantial size including dozens
>>> of fields, which then had nested records. It did not scale to write out AVSC
>>> definitions for the data model.
>>>
>>>
>>> On Thu, Feb 20, 2014 at 8:28 PM, Software Dev <static.void.dev@gmail.com>
>>> wrote:
>>>>
>>>> Thanks for the input. I'm guessing then the above problem can only
>>>> solved with IDL and not AVSC?
>>>>
>>>>
>>>> On Thu, Feb 20, 2014 at 11:45 AM, Lewis John Mcgibbney
>>>> <lewis.mcgibbney@gmail.com> wrote:
>>>>>
>>>>> Hey,
>>>>> Did you check out the IDL documentation?
>>>>> http://avro.apache.org/docs/current/idl.html
>>>>> I had similar data modeling issues a while back and this helped out A
>>>>> LOT.
>>>>> hth
>>>>>
>>>>>
>>>>> On Thu, Feb 20, 2014 at 6:58 PM, Software Dev
>>>>> <static.void.dev@gmail.com> wrote:
>>>>>>
>>>>>> Is there anyway to include the fields of another schema into our
>>>>>> schema WITHOUT it creating a nested record?
>>>>>>
>>>>>>
>>>>>> {
>>>>>>     "type": "record",
>>>>>>     "name": "Parent",
>>>>>>     "fields" : [
>>>>>>         {
>>>>>>             "name": "foo",
>>>>>>             "type": "string"
>>>>>>         }
>>>>>>     ]
>>>>>> }
>>>>>>
>>>>>> {
>>>>>>     "type": "record",
>>>>>>     "name": "Child",
>>>>>>     "fields" : [
>>>>>>         {
>>>>>>             "name": "bar",
>>>>>>             "type": "string"
>>>>>>         },
>>>>>> // I dont want it nested like this
>>>>>> //        {
>>>>>> //            "name": "parent",
>>>>>> //            "type": "Parent"
>>>>>> //        }
>>>>>>     ]
>>>>>> }
>>>>>>
>>>>>> So in this example is there a way to have child include both the "bar"
>>>>>> field as well as "foo" without it nested under parent?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lewis
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Lewis
>>
>>
>


-- 
Harsh J