avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Smith <lenov...@gmail.com>
Subject Re: custom attributes
Date Thu, 07 Jan 2016 10:52:03 GMT
Ok,

looks like my fail, the problem seems to be inside
https://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/AvroStorage.java


When I try to dump data using Pig AvroStorage all the additional attributes
are removed from AVRO schema.





On Thu, Jan 7, 2016 at 10:53 AM, John Smith <lenovomi@gmail.com> wrote:

> Hi,
>
> Im using parser() method (
> https://avro.apache.org/docs/1.7.7/api/java/org/apache/avro/Schema.Parser.html#parse%28java.lang.String%29).
>
>
> The question is if i have to write my own parser() method, extending the
> one from the AVRO api or if there is any other elegant solution to
> implement it.
>
>
> Thank you
>
>
> On Thu, Jan 7, 2016 at 8:11 AM, John Smith <lenovomi@gmail.com> wrote:
>
>> Hi Peter,
>>
>> sorry it was mistype, it should be
>>
>> {
>>         "type" :"record",
>>         "name": "Test",
>>         "fields": [
>>             {
>>             "name": "ROW_ID",
>>             "title": "row identifier ",
>>             "type": "integer",
>>
>>
>> *            "description": "something",,            "ValidityDate":
>> "2015-01-01",            "ValidityOption": "Delete"*
>>             },
>> ...
>>
>>
>> But how you add your own attributes, in your case "primarykey":"true”
>>
>>
>> Thanks
>>
>> On Thu, Jan 7, 2016 at 1:16 AM, Peter Lappo <peter.lappo@gmail.com>
>> wrote:
>>
>>> Yes you can but be careful with the trailing comma - avro parser is
>>> quite strict and will fail
>>>
>>> *"ValidityOption": "Delete",*
>>>
>>>
>>> I use
>>> {"name":"CUST_ID","type":"string","primarykey":"true”}
>>> and define the custom attribute in the avsc file
>>>
>>>
>>> On 6 Jan 2016, at 21:11, John Smith <lenovomi@gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> Im wondering if its allowed to define custom attributes as part of the
>>> AVRO schema, e.g.
>>>
>>>
>>> {
>>>         "type" :"record",
>>>         "name": "Test",
>>>         "fields": [
>>>             {
>>>             "name": "ROW_ID",
>>>             "title": "row identifier ",
>>>             "type": "integer",
>>>
>>>
>>> *            "description": "something",,            "ValidityDate":
>>> "2015-01-01",            "ValidityOption": "Delete",*
>>>             },
>>> ...
>>>
>>>
>>> Can I define those three new attributes *description, **ValidityDate, **ValidityOption
>>> *that extends description of the record "ROW_ID"?
>>>
>>>
>>> Thank you
>>>
>>>
>>>
>>>
>>
>

Mime
View raw message