nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ron Chittaro (Jira)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-6654) Stack overflow with self referencing avro schema
Date Thu, 12 Sep 2019 18:44:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-6654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928817#comment-16928817
] 

Ron Chittaro commented on NIFI-6654:
------------------------------------

Validating a fix for this.

> Stack overflow with self referencing avro schema
> ------------------------------------------------
>
>                 Key: NIFI-6654
>                 URL: https://issues.apache.org/jira/browse/NIFI-6654
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.9.2
>            Reporter: Ron Chittaro
>            Priority: Major
>
> We (and our customers, thus why I marked this a blocker as we cannot proceed with 1.9.2
and in some cases product depolyment at all) have run into a stack overflow exception with
the 'UpdateRecord' processor when upgrading from Nifi 1.7 to 1.9.2. I did some digging and
it happens when:
>  * A schema comparison operation is performed, AND
>  * the schema in question is self referential
> I was a bit confused initially as I found that a unit test case for this already existed
and was passing when I checked out and built 1.9.2. The test case is:
> {quote}testHashCodeAndEqualsWithSelfReferencingSchema() from TestSimpleRecordSchema.java
located in: nifi\nifi-commons\nifi-record\src\test\java\org\apache\nifi\serialization
> {quote}
> However, if you dig a bit into this test case it is passing because the two schemas being
compared contain the same memory references for the fields within them, thus comparisons do
not exercise completely the underlying equals() operators of the objects contained in the
schema. See bold below:
> {quote}final SimpleRecordSchema schema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
> final List<RecordField> personFields = new ArrayList<>();
>  personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("sibling", RecordFieldType.RECORD.getRecordDataType(schema)));
> *schema.setFields(personFields);*
> schema.hashCode();
>  assertTrue(schema.equals(schema));
> final SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
>  *secondSchema.setFields(personFields);*
>  assertTrue(schema.equals(secondSchema));
>  assertTrue(secondSchema.equals(schema));
> {quote}
>  
> To reproduce the stack overflow I wrote the following test cases which I believe behaves
more like it would in the 'UpdateProcessor' (or anywhere else a schema comparison is happening)
where the schema object and all objects within are completely different memory references.
>  
> @Test
> public void testSchemaCompareSelfRef() {
>  // Set up first schema
>  final SimpleRecordSchema personSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
>  final List<RecordField> personFields = new ArrayList<>();
>  personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(personSchema)));
>  personSchema.setFields(personFields);
>  // Set up second schema. Must be completely separate set of objects otherwise overloaded
comparison
>  // operators (particularly that from the SimpleRecordSchema class) will not be called.
>  SimpleRecordSchema secondSchema = new SimpleRecordSchema(SchemaIdentifier.EMPTY);
>  personFields.clear();
>  personFields.add(new RecordField("name", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("address", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("SIN", RecordFieldType.STRING.getDataType()));
>  personFields.add(new RecordField("PersonalData", RecordFieldType.RECORD.getRecordDataType(secondSchema)));
>  secondSchema.setFields(personFields);
>  assertTrue(personSchema.equals(secondSchema));
> }
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message