kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-2483) Add support for batch/scheduled Copycat connectors
Date Thu, 27 Aug 2015 17:11:46 GMT
Ewen Cheslack-Postava created KAFKA-2483:

             Summary: Add support for batch/scheduled Copycat connectors
                 Key: KAFKA-2483
                 URL: https://issues.apache.org/jira/browse/KAFKA-2483
             Project: Kafka
          Issue Type: Sub-task
          Components: copycat
            Reporter: Ewen Cheslack-Postava
            Assignee: Ewen Cheslack-Postava

A few connectors may not perform well if run continuously; for example, HDFS may not handle
a task holding a file open for very long periods of time well.

These connectors will work better if they can schedule themselves to be executed periodically.
Note that this cannot currently be implemented by the connector itself because in sink connectors
get data delivered to them as it streams in. However, it might be possible to make connectors
handle this themselves given the combination of KAFKA-2481 and KAFKA-2482 would make it possible,
if inconvenient, to implement this in the task itself.

This message was sent by Atlassian JIRA

View raw message