kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3054) Connect Herder fail forever if sent a wrong connector config or task config
Date Mon, 11 Jan 2016 18:07:39 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092392#comment-15092392

Ewen Cheslack-Postava commented on KAFKA-3054:

[~jinxing6042@126.com] We do want to catch them, but probably don't want to just swallow them.
Although that might be a short-term solution for this specific problem. We don't do a good
job of tracking connector/task status in Connect right now. We'll need to track this information
(and also expose it via the REST API, and allow control via APIs like suggested in KAFKA-2370).
I know [~hachikuji] is also working on KAFKA-2886 now, which also faces the same problem --
we can sort of half fix the issue before we have support for tracking status info.

I'd say a good short term solution would be to catch other exceptions and at a minimum log
it at ERROR level. I haven't thought through the types of exceptions that might be generated,
but it's possible we'll want to treat different exceptions somewhat differently (e.g. if they
throw a ConnectException, the connector may have hit an issue, but is behaving well; if they
throw anything that we can only classify as Throwable, we probably want to treat that as a
bug in the connector itself and complain more loudly about it in the log). Then you might
want to file a follow-up JIRA to make sure we don't lose track of that status change when
we have support for tracking it.

> Connect Herder fail forever if sent a wrong connector config or task config
> ---------------------------------------------------------------------------
>                 Key: KAFKA-3054
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3054
>             Project: Kafka
>          Issue Type: Bug
>          Components: copycat
>    Affects Versions:
>            Reporter: jin xing
>            Assignee: jin xing
> Connector Herder throws ConnectException and shutdown if sent a wrong config, restarting
herder will keep failing with the wrong config; It make sense that herder should stay available
when start connector or task failed; After receiving a delete connector request, the herder
can delete the wrong config from "config storage"

This message was sent by Atlassian JIRA

View raw message