kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leonardo Toledo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3910) Cyclic schema support in ConnectSchema and SchemaBuilder
Date Wed, 07 Feb 2018 22:08:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356145#comment-16356145
] 

Leonardo Toledo commented on KAFKA-3910:
----------------------------------------

Hi, someone can tell me if there are plans to fix and close this issue please? 

Thanks in advance. 

> Cyclic schema support in ConnectSchema and SchemaBuilder
> --------------------------------------------------------
>
>                 Key: KAFKA-3910
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3910
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>    Affects Versions: 0.10.0.0
>            Reporter: John Hofman
>            Assignee: Shikhar Bhushan
>            Priority: Major
>
> Cyclic schema's are not supported by ConnectSchema or SchemaBuilder. Subsequently the
AvroConverter (confluentinc/schema-registry) hits a stack overflow when converting a cyclic
avro schema, e.g:
> {code}
> {"type":"record", "name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}
> {code}
> This is a blocking issue for all connectors running on the connect framework with data
containing cyclic references. The AvroConverter cannot support cyclic schema's until the underlying
ConnectSchema and SchemaBuilder do.
> To reproduce the stack-overflow (Confluent-3.0.0):
> Produce some cyclic data:
> {code}
> bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test --property
value.schema='{"type":"record", "name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}'
> {"value":1,"next":null} 
> {"value":1,"next":{"list":{"value":2,"next":null}}}
> {code}
> Then try to consume it with connect:
> {code:title=connect-console-sink.properties}
> name=local-console-sink 
> connector.class=org.apache.kafka.connect.file.FileStreamSinkConnector 
> tasks.max=1 
> topics=test
> {code}
> {code}
> ./bin/connect-standalone ./etc/schema-registry/connect-avro-standalone.properties connect-console-sink.properties
 
> … start up logging … 
> java.lang.StackOverflowError 
>  at org.apache.avro.JsonProperties.getJsonProp(JsonProperties.java:54) 
>  at org.apache.avro.JsonProperties.getProp(JsonProperties.java:45) 
>  at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1055) 
>  at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103) 
>  at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137) 
>  at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103) 
>  at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message