kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Urban (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-13253) Kafka Connect losing task (re)configuration when connector name has special characters
Date Mon, 30 Aug 2021 15:56:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Urban resolved KAFKA-13253.
----------------------------------
    Resolution: Duplicate

Same issue as KAFKA-9747 - that one already has a fix under review

> Kafka Connect losing task (re)configuration when connector name has special characters
> --------------------------------------------------------------------------------------
>
>                 Key: KAFKA-13253
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13253
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 2.7.1
>            Reporter: David Dufour
>            Priority: Major
>
> When not leader, DistributedHerder.reconfigureConnector() forwards the task configuration
to the leader as follow:
> {quote}String reconfigUrl = RestServer.urlJoin(leaderUrl, "/connectors/" + connName +
"/tasks");
> log.trace("Forwarding task configurations for connector {} to leader", connName);
> RestClient.httpRequest(reconfigUrl, "POST", null, rawTaskProps, null, config, sessionKey,
requestSignatureAlgorithm);
> {quote}
> The problem is that if the connector name contains some special characters, such as '<',
'>'... they need to be 'URLEncoded' appropriately, otherwise an uncatched exception is
raised in RestClient and the forward is lost.
> Here is the kind of exception we can catch by adding the necessary code in RestClient:
> {quote}java.lang.IllegalArgumentException: Illegal character in path at index 51: [http://10.224.0.15:8083/connectors/mirror1-cluster-]>mirror2-cluster.MirrorHeartbeatConnector/tasks
> {quote}
> An additional catch() should be added in RestClient.httpRequest(), here:
> {quote}{{catch (IOException | InterruptedException | TimeoutException | ExecutionException
e) {}}
>     log.error("IO error forwarding REST request: ", e);
>     {{throw new ConnectRestException(Response.Status.INTERNAL_SERVER_ERROR, "IO Error
trying to forward REST request: " + e.getMessage(), e);}}
> {{} finally {}}
> {quote}
> to catch all other exceptions because without, this kind of problem is completly silent.
> To reproduce:
>  * start 2 kafka clusters
>  * start a kafka connect (distributed) with at least 2 nodes
>  * start an HeartbeatConnector with name "cluster1->cluster2"
> If the node which generated the task is not the leader (not systematic), it will forward
the creation to the leader and it will be lost. As a result, the connector will stay in RUNNING
state but without any task.
> Problem not easy to reproduce, it is important to start with empty connect topics to
reproduce more easily



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message