flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Stack overflow from self referencing Avro schema
Date Thu, 10 Mar 2016 17:06:00 GMT
The following issue should track that.
https://issues.apache.org/jira/browse/FLINK-3602

@Niels: Thanks for looking into this. At this point, I think it may
actually be a Flink issue, since it concerns the interaction of Avro and
Flink's TypeInformation.

On Thu, Mar 10, 2016 at 6:00 PM, Stephan Ewen <sewen@apache.org> wrote:

> Hi!
>
> I think that is a TypeExtractor bug. It may actually be a bug for all
> recursive types.
> Let's check this and come up with a fix...
>
> Greetings,
> Stephan
>
>
> On Thu, Mar 10, 2016 at 4:11 PM, David Kim <
> david.kim@braintreepayments.com> wrote:
>
>> Hello!
>>
>> Just wanted to check up on this again. Has anyone else seen this before
>> or have any suggestions?
>>
>> Thanks!
>> David
>>
>> On Tue, Mar 8, 2016 at 12:12 PM, David Kim <
>> david.kim@braintreepayments.com> wrote:
>>
>>> Hello all,
>>>
>>> I'm running into a StackOverflowError using flink 1.0.0. I have an Avro
>>> schema that has a self reference. For example:
>>>
>>> item.avsc
>>>
>>> {
>>>
>>>   "namespace": "..."
>>>
>>>   "type": "record"
>>>   "name": "Item",
>>>   "fields": [
>>>     {
>>>       "name": "parent"
>>>       "type": ["null, "Item"]
>>>     }
>>>   ]
>>> }
>>>
>>>
>>> When running my flink job, I'm running into the follow error:
>>>
>>> Exception in thread "Thread-94" java.lang.StackOverflowError
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.countTypeInHierarchy(TypeExtractor.java:1105)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1397)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.<init>(AvroTypeInfo.java:48)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.<init>(AvroTypeInfo.java:48)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1394)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.privateGetForClass(TypeExtractor.java:1319)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.createTypeInfoWithTypeHierarchy(TypeExtractor.java:609)
>>> 	at org.apache.flink.api.java.typeutils.TypeExtractor.analyzePojo(TypeExtractor.java:1531)
>>> 	at org.apache.flink.api.java.typeutils.AvroTypeInfo.generateFieldsFromAvroSchema(AvroTypeInfo.java:53)
>>>
>>>
>>> Interestingly if I change the type to an Avro array in the schema, this
>>> error is not thrown.
>>>
>>> Thanks!
>>> David
>>>
>>
>>
>>
>> --
>> Note: this information is confidential. It is prohibited to share, post
>> online or otherwise publicize without Braintree's prior written consent.
>>
>
>

Mime
View raw message