kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Egerton <chr...@confluent.io>
Subject Re: [DISCUSS] KIP-449: Add connector contexts to Connect worker logs
Date Mon, 15 Apr 2019 17:54:40 GMT
Hi Randall,

Thanks for the KIP. Debugging Connect workers definitely becomes harder as
the number of connectors and tasks increases, and your proposal would
simplify the process of sifting through logs and finding relevant
information faster and more accurately.

I have a couple small comments:

First--I believe the example snippet in your KIP under the "Public
Interfaces" header is inaccurate:
`[my-connector|worker]` - used on log messages where the Connect worker is
validating the configuration for or starting/stopping the
"local-file-source" connector via the SourceConnector / SinkConnector
implementation methods.
`[my-connector|task-0]` - used on log messages where the Connect worker is
executing task 0 of the "local-file-source" connector, including calling
any of the SourceTask / SinkTask implementation methods, processing the
messages for/from the task, and calling the task's producer/consumer.
`[my-connector|task-0|offsets]` - used on log messages where the Connect
worker is committing source offsets for task 0 of the "local-file-source"
connector.
The sample contexts mention "my-connector" but their explanations say that
they correspond to "local-file-source"; shouldn't the two align?

Second--I'm unclear on whether we expect (or want to encourage) developers
to manipulate the "connector.context" MDC key themselves, from with
connectors, transforms, etc. If we want to encourage this (in order to make
debugging even easier, which would align with the intent behind this KIP),
we may want to expose the LoggingContext class in the Connect API package
and expand on it so that users can set the context themselves. This would
be especially helpful in connectors with multithreaded logic. However, if
that would expand the scope of this KIP too much I think we could afford to
address that later.

Cheers,

Chris

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message