avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ken Krugler <kkrugler_li...@transpac.com>
Subject Re: New name requirements for Avro 1.5
Date Fri, 10 Jun 2011 23:16:15 GMT

On Jun 10, 2011, at 4:01pm, Scott Carey wrote:

> Ok, there was a bug in 1.4 that allowed for invalid names.  Sounds like we
> need something that allows reading some older files.
> Please open a JIRA ticket explaining the issue, we can discuss
> solutions/workarounds there.

https://issues.apache.org/jira/browse/AVRO-838

Thanks,

-- Ken


> 
> On 6/10/11 1:48 PM, "Ken Krugler" <kkrugler_lists@transpac.com> wrote:
> 
>> I'd run into a similar issue, where I used a record name with a "-" in it
>> when creating files with Avro 1.4.
>> 
>> So I've got files that are valid/readable with 1.4, but seem to be
>> unusable with 1.5.
>> 
>> -- Ken
>> 
>> On Jun 10, 2011, at 12:32pm, Matt Pouttu-Clarke wrote:
>> 
>>> OK, thanks.  This is a change from Avro 1.4 which will make older Avro
>>> files
>>> unreadable by 1.5 until the field names are changed.
>>> 
>>> Can we get away with "tweeking" the field name in the JSON meta data
>>> portion
>>> of the file, or do we have to re-write the whole file again with the new
>>> meta data?
>>> 
>>> On 6/10/11 12:23 PM, "Douglas Creager" <dcreager@dcreager.net> wrote:
>>> 
>>>>> I am using Avro for processing/encoding billions of rows using
>>>>> Hadoop.  When
>>>>> I attempted to upgrade to Avro 1.5 I got an error about illegal field
>>>>> names.
>>>>> I notice this case happens when I use a O-¹ character or a O.¹ in a
>>>>> field
>>>>> name.  
>>>>> 
>>>>> Are there any other restrictions on fields names I need to know when
>>>>> upgrading?
>>>> 
>>>> You can find the full details in the Avro spec [1].  It's basically
>>>> just like
>>>> the identifier restrictions in most programming languages.  Names can
>>>> only
>>>> contain letters, numbers, and underscores, and can't start with a
>>>> number.
>>>> Dots are used to created namespaces, just like fully-qualified Java
>>>> identifiers.
>>>> 
>>>> [1] http://avro.apache.org/docs/current/spec.html
>>>> 
>>>> cheers
>>>> doug
>>> 
>>> 
>>> iCrossing Privileged and Confidential Information
>>> This email message is for the sole use of the intended recipient(s) and
>>> may contain confidential and privileged information of iCrossing. Any
>>> unauthorized review, use, disclosure or distribution is prohibited. If
>>> you are not the intended recipient, please contact the sender by reply
>>> email and destroy all copies of the original message.
>>> 
>>> 
>> 
>> --------------------------
>> Ken Krugler
>> +1 530-210-6378
>> http://bixolabs.com
>> custom data mining solutions
>> 
>> 
>> 
>> 
>> 
>> 
> 

--------------------------
Ken Krugler
+1 530-210-6378
http://bixolabs.com
custom data mining solutions







Mime
View raw message