hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Durfey <sjdur...@gmail.com>
Subject Hive and Schema Registry
Date Thu, 12 Oct 2017 13:52:38 GMT
Recently my team has opened a discussion with Confluent [1] in regards to
the schema registry being used to serve up avro schemas for the Hive
AvroSerDe to make use of through 'avro.schema.url' config. Originally we
were hoping to just get a REST endpoint that returns just the schema to
avoid making any changes to the AvroSerDe. The confluent rest endpoint
today returns the avro schema embedded as an attribute inside a json
response [2] which makes it unusable by the AvroSerDe.

I wanted to reach out to the community to talk about the possibility of
enhancing the AvroSerDe to be able to make use of JSON responses returned
from the configured URL. One of the possibilities mentioned in the
confluent github issue was to add in a new (optional) configuration to
identify the field within the JSON response and the AvroSerDe, and if set
use that config to retrieve the schema from that attribute.

We're open to other suggestions and would be happy to contribute the patch
back to hive for whatever design is settled on.

- Stephen


[1] https://github.com/confluentinc/schema-registry/issues/629
[2]
https://docs.confluent.io/current/schema-registry/docs/api.html#get--subjects-(string-
subject)-versions-(versionId- version)

Mime
View raw message