nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wesley L Lawrence (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
Date Tue, 01 Aug 2017 15:31:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109082#comment-16109082
] 

Wesley L Lawrence edited comment on NIFI-4215 at 8/1/17 3:30 PM:
-----------------------------------------------------------------

[~markap14] Thanks for adding a second eye to this code change.

I see there error that was added into AvroTypeUtil (on line 357, createSchema(...) ). Sorry
about that, I'll get that fixed back to 'schemaId'.

I was hesitant to change the immutability of SimleRecordSchema, but there needs to be a way
to create a place holder for the in-progress record parsing, so that sub-records of the same
type don't create a parsing loop.

Looking back with a fresh set of eyes; I could create a 'SimpleRecordSchemaBuilder' that's
mutable, and once parsing is complete, returns the immutable 'SimpleRecordSchema'.

Any other ideas or feedback are welcome, and I'll get around to addressing this as soon as
possible.

--Wes




was (Author: wesleylawrence):
[~markap14] Thanks for adding a second eye to this code change.

I see there error that was added into AvroTypeUtil (on line 357, createSchema(...) ).

I was hesitant to change the immutability of SimleRecordSchema, but there needs to be a way
to create a place holder for the in-progress record parsing, so that sub-records of the same
type don't create a parsing loop.

Looking back with a fresh set of eyes; I could create a 'SimpleRecordSchemaBuilder' that's
mutable, and once parsing is complete, returns the immutable 'SimpleRecordSchema'.

Any other ideas or feedback are welcome, and I'll get around to addressing this as soon as
possible.

--Wes



> Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow
exception
> --------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-4215
>                 URL: https://issues.apache.org/jira/browse/NIFI-4215
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>            Reporter: Wesley L Lawrence
>            Priority: Blocker
>             Fix For: 1.4.0
>
>         Attachments: nifi-4215.patch
>
>
> Noticed this while attempting to use the AvroSchemaRegsitry with some complex schema.
Boiled down, Avro lets you define a schema such as;
> {code}
> { 
>   "namespace": "org.apache.nifi.testing", 
>   "name": "CompositRecord", 
>   "type": "record", 
>   "fields": [ 
>     { 
>       "name": "id", 
>       "type": "int" 
>     }, 
>     { 
>       "name": "value", 
>       "type": "string" 
>     }, 
>     { 
>       "name": "parent", 
>       "type": [
>         "null",
>         "CompositRecord"
>       ]
>     } 
>   ] 
> }
> {code}
> The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and generate a
stackoverflow exception.
> I've whipped up a fix, tested it out in 1.4.0, and am just running through the contrib
build before I submit a patch.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message