camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tim.jones" <tim.jones....@gmail.com>
Subject Re: Camel-Kafka avro schema registry & (de)serializers configuration in spring-boot
Date Tue, 30 May 2017 20:28:02 GMT
Claus

Thanks for taking the time to respond to this camel newbie.

The schema registry url is the host & port for the schema registry, which
allows the KafkaAvroSerializer
(io.confluent.kafka.serializers.KafkaAvroSerializer) to look up the schema
used to serialize and deserialise an avro message, to avoid the need to
include this information with each message.  See
http://docs.confluent.io/current/app-development.html#java  and
http://docs.confluent.io/current/schema-registry/docs/intro.html
<http://docs.confluent.io/current/schema-registry/docs/intro.html#>

There is another thread on camel-users where some other people have been
trying get around the “Schema Registry Url problem”
http://camel.465427.n5.nabble.com/Camel-2-17-Kafka-SSL-Producer-and-Consumer-td5779947.html
which
seems to suggest that the way to achieve this is to implement custom
KafkaAvroDeserializer (& by extension also a  KafkaAvroDeserializer) in
which to pass in the URL of the schema registry, which seems rather
complicated, but perhaps this is the only way?

Regards,

Tim







On 30 May 2017 at 18:53, Claus Ibsen-2 [via Camel] <
ml+s465427n5801389h78@n5.nabble.com> wrote:

> The github docs are the up to date docs, so look there.
>
>
> What is schema.registry.url,  ?
>
>
>
> For configuring Camel uris, then see
> http://camel.apache.org/uris
>
> Where you can use the # syntax to refer to beans in the spring boot
> bean registry
>
> Then to refer to a custom bean in the registry then use spring boot
> @Bean("nameGoesHere") on a method that crates and return the bean
> which is that avro serializer or whatever you need, to return a bean
> with that name, which you can then use #nameGoesHere style in the
> camel uri to refer to.
>
>
> On Tue, May 30, 2017 at 7:27 PM, tim.jones <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=5801389&i=0>> wrote:
>
> > Claus
> >
> > Sorry to be dense, but which camel component should I be looking at and
> > where?
> >
> > If you meant the kafka component, I could not see anything which
> mentioned
> > the schema registry in its docs at:
> > https://github.com/apache/camel/blob/master/components/
> camel-kafka/src/main/docs/kafka-component.adoc
> >
> > I do see a reference to “configuration (common)” listed first in the 6
> > Kafka component options - is this mechanism for introduction of other
> > config options such as the the schema.registry.url into the endpoint
> > configuration - if so, how do you configure the component, as opposed to
> > the endpoint?
> >
> > Regards,
> >
> > Tim
> >
> > On 30 May 2017 at 17:48, Claus Ibsen-2 [via Camel] <
> > [hidden email] <http:///user/SendEmail.jtp?type=node&node=5801389&i=1>>
> wrote:
> >
> >> See the Camel component docs via github
> >> https://github.com/apache/camel/tree/master/components#components
> >>
> >> On Tue, May 30, 2017 at 6:19 PM, tim.jones <[hidden email]
> >> <http:///user/SendEmail.jtp?type=node&node=5801387&i=0>> wrote:
> >>
> >> > Hi
> >> >
> >> > Using the confluent schema registry for the avro schemas and wanted
> to
> >> use
> >> > camel-kafka (with spring-boot) , but can't work out how to introduce
> the
> >> > schema.registry.url, as it is not a URI option.
> >> >
> >> > How can I include this property, such that it is picked up both by
> >> > Camel-kafka producers and consumers properly?
> >> >
> >> > Is there also something I should know about correctly specifying the
> >> kafka
> >> > avro serializers in the serializerClass &  keySerializerClass in
> >> camel-kafka
> >> > producer options and the  keyDeserializer & valueDeserializer in the
> >> > consumer options?
> >> >
> >> > (there does not seem to be a one-to-one mapping between these
> >> camel-kafka
> >> > options (i.e.  camel.apache.org Kafka-ProducerOptions
> >> > <http://camel.apache.org/kafka.html#Kafka-ProducerOptions(Camel2.
> >> 17ornewer)>
> >> > ) and the config options on the kafka website i.e.  kafka.apache.org
> >> > ewproducerconfigs
> >> > <http://kafka.apache.org/documentation.html#newproducerconfigs> 
)
> >> >
> >> >
> >> > Thanks for your help,
> >> >
> >> > Tim
> >> >
> >> >
> >> >
> >> > --
> >> > View this message in context: http://camel.465427.n5.nabble.
> >> com/Camel-Kafka-avro-schema-registry-de-serializers-
> >> configuration-in-spring-boot-tp5801381.html
> >> > Sent from the Camel - Users mailing list archive at Nabble.com.
> >>
> >>
> >>
> >> --
> >> Claus Ibsen
> >> -----------------
> >> http://davsclaus.com @davsclaus
> >> Camel in Action 2: https://www.manning.com/ibsen2
> >>
> >>
> >> ------------------------------
> >> If you reply to this email, your message will be added to the
> discussion
> >> below:
> >> http://camel.465427.n5.nabble.com/Camel-Kafka-avro-schema-
> >> registry-de-serializers-configuration-in-spring-boot-
> >> tp5801381p5801387.html
> >> To unsubscribe from Camel-Kafka avro schema registry & (de)serializers
> >> configuration in spring-boot, click here
> >> <
> >> .
> >> NAML
> >> <http://camel.465427.n5.nabble.com/template/
> NamlServlet.jtp?macro=macro_viewer&id=instant_html%
> 21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> nabble.view.web.template.NodeNamespace&breadcrumbs=
> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> >>
> >
> >
> >
> >
> > --
> > View this message in context: http://camel.465427.n5.nabble.
> com/Camel-Kafka-avro-schema-registry-de-serializers-
> configuration-in-spring-boot-tp5801381p5801388.html
> > Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://camel.465427.n5.nabble.com/Camel-Kafka-avro-schema-
> registry-de-serializers-configuration-in-spring-boot-
> tp5801381p5801389.html
> To unsubscribe from Camel-Kafka avro schema registry & (de)serializers
> configuration in spring-boot, click here
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5801381&code=dGltLmpvbmVzLjAwMUBnbWFpbC5jb218NTgwMTM4MXwyNzg0MjcyMDc=>
> .
> NAML
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Kafka-avro-schema-registry-de-serializers-configuration-in-spring-boot-tp5801381p5801394.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message