flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Kim <david....@braintreepayments.com>
Subject Re: Stack overflow from self referencing Avro schema
Date Fri, 11 Mar 2016 16:11:12 GMT
Thanks Stephan! :)

On Thu, Mar 10, 2016 at 11:06 AM, Stephan Ewen <sewen@apache.org> wrote:

> The following issue should track that.
> https://issues.apache.org/jira/browse/FLINK-3602
>
> @Niels: Thanks for looking into this. At this point, I think it may
> actually be a Flink issue, since it concerns the interaction of Avro and
> Flink's TypeInformation.
>
> On Thu, Mar 10, 2016 at 6:00 PM, Stephan Ewen <sewen@apache.org> wrote:
>
>> Hi!
>>
>> I think that is a TypeExtractor bug. It may actually be a bug for all
>> recursive types.
>> Let's check this and come up with a fix...
>>
>> Greetings,
>> Stephan
>>
>>
>> On Thu, Mar 10, 2016 at 4:11 PM, David Kim <
>> david.kim@braintreepayments.com> wrote:
>>
>>> Hello!
>>>
>>> Just wanted to check up on this again. Has anyone else seen this before
>>> or have any suggestions?
>>>
>>> Thanks!
>>> David
>>>
>>> On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
>>> david.kim@braintreepayments.com> wrote:
>>>
>>>> Hello all,
>>>>
>>>> I'm running into a StackOverflowError using flink 1.0.0. I have an Avro
>>>> schema that has a self reference. For example:
>>>>
>>>> item.avsc
>>>>
>>>> {
>>>>
>>>>   "namespace": "..."
>>>>
>>>>   "type": "record"
>>>>   "name": "Item",
>>>>   "fields": [
>>>>     {
>>>>       "name": "parent"
>>>>       "type": ["null, "Item"]
>>>>     }
>>>>   ]
>>>> }
>>>>
>>>>
>>>> When running my flink job, I'm running into the follow error:
>>>>
>>>> Exception in thread "Thread-94" java.lang.StackOverflowError
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.<init>(AvroTypeInfo.java:48)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.<init>(AvroTypeInfo.java:48)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>>>
>>>>
>>>> Interestingly if I change the type to an Avro array in the schema,
>>>> this error is not thrown.
>>>>
>>>> Thanks!
>>>> David
>>>>
>>>
>>>
>>>
>>> --
>>> Note: this information is confidential. It is prohibited to share, post
>>> online or otherwise publicize without Braintree's prior written consent.
>>>
>>
>>
>


-- 
Note: this information is confidential. It is prohibited to share, post
online or otherwise publicize without Braintree's prior written consent.

Mime
View raw message