flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Kim <david....@braintreepayments.com>
Subject Re: Stack overflow from self referencing Avro schema
Date Tue, 22 Mar 2016 14:42:20 GMT
Hi Robert!

Thank you! :)

David

On Tue, Mar 22, 2016 at 7:59 AM, Robert Metzger <rmetzger@apache.org> wrote:

> Hey David,
>
> FLINK-3602 has been merged to master.
>
> On Fri, Mar 11, 2016 at 5:11 PM, David Kim <
> david.kim@braintreepayments.com> wrote:
>
>> Thanks Stephan! :)
>>
>> On Thu, Mar 10, 2016 at 11:06 AM, Stephan Ewen <sewen@apache.org> wrote:
>>
>>> The following issue should track that.
>>> https://issues.apache.org/jira/browse/FLINK-3602
>>>
>>> @Niels: Thanks for looking into this. At this point, I think it may
>>> actually be a Flink issue, since it concerns the interaction of Avro and
>>> Flink's TypeInformation.
>>>
>>> On Thu, Mar 10, 2016 at 6:00 PM, Stephan Ewen <sewen@apache.org> wrote:
>>>
>>>> Hi!
>>>>
>>>> I think that is a TypeExtractor bug. It may actually be a bug for all
>>>> recursive types.
>>>> Let's check this and come up with a fix...
>>>>
>>>> Greetings,
>>>> Stephan
>>>>
>>>>
>>>> On Thu, Mar 10, 2016 at 4:11 PM, David Kim <
>>>> david.kim@braintreepayments.com> wrote:
>>>>
>>>>> Hello!
>>>>>
>>>>> Just wanted to check up on this again. Has anyone else seen this
>>>>> before or have any suggestions?
>>>>>
>>>>> Thanks!
>>>>> David
>>>>>
>>>>> On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
>>>>> david.kim@braintreepayments.com> wrote:
>>>>>
>>>>>> Hello all,
>>>>>>
>>>>>> I'm running into a StackOverflowError using flink 1.0.0. I have an
>>>>>> Avro schema that has a self reference. For example:
>>>>>>
>>>>>> item.avsc
>>>>>>
>>>>>> {
>>>>>>
>>>>>>   "namespace": "..."
>>>>>>
>>>>>>   "type": "record"
>>>>>>   "name": "Item",
>>>>>>   "fields": [
>>>>>>     {
>>>>>>       "name": "parent"
>>>>>>       "type": ["null, "Item"]
>>>>>>     }
>>>>>>   ]
>>>>>> }
>>>>>>
>>>>>>
>>>>>> When running my flink job, I'm running into the follow error:
>>>>>>
>>>>>> Exception in thread "Thread-94" java.lang.StackOverflowError
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.<init>(AvroTypeInfo.java:48)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.<init>(AvroTypeInfo.java:48)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>>>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>>>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>>>>>
>>>>>>
>>>>>> Interestingly if I change the type to an Avro array in the schema,
>>>>>> this error is not thrown.
>>>>>>
>>>>>> Thanks!
>>>>>> David
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Note: this information is confidential. It is prohibited to share,
>>>>> post online or otherwise publicize without Braintree's prior written
>>>>> consent.
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Note: this information is confidential. It is prohibited to share, post
>> online or otherwise publicize without Braintree's prior written consent.
>>
>
>


-- 
Note: this information is confidential. It is prohibited to share, post
online or otherwise publicize without Braintree's prior written consent.

Mime
View raw message