From dev-return-39770-apmail-kafka-dev-archive=kafka.apache.org@kafka.apache.org Thu Aug 27 17:11:46 2015 Return-Path: X-Original-To: apmail-kafka-dev-archive@www.apache.org Delivered-To: apmail-kafka-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D30751867E for ; Thu, 27 Aug 2015 17:11:46 +0000 (UTC) Received: (qmail 36424 invoked by uid 500); 27 Aug 2015 17:11:46 -0000 Delivered-To: apmail-kafka-dev-archive@kafka.apache.org Received: (qmail 36336 invoked by uid 500); 27 Aug 2015 17:11:46 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 36325 invoked by uid 99); 27 Aug 2015 17:11:46 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Aug 2015 17:11:46 +0000 Date: Thu, 27 Aug 2015 17:11:46 +0000 (UTC) From: "Ewen Cheslack-Postava (JIRA)" To: dev@kafka.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (KAFKA-2483) Add support for batch/scheduled Copycat connectors MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Ewen Cheslack-Postava created KAFKA-2483: -------------------------------------------- Summary: Add support for batch/scheduled Copycat connectors Key: KAFKA-2483 URL: https://issues.apache.org/jira/browse/KAFKA-2483 Project: Kafka Issue Type: Sub-task Components: copycat Reporter: Ewen Cheslack-Postava Assignee: Ewen Cheslack-Postava A few connectors may not perform well if run continuously; for example, HDFS may not handle a task holding a file open for very long periods of time well. These connectors will work better if they can schedule themselves to be executed periodically. Note that this cannot currently be implemented by the connector itself because in sink connectors get data delivered to them as it streams in. However, it might be possible to make connectors handle this themselves given the combination of KAFKA-2481 and KAFKA-2482 would make it possible, if inconvenient, to implement this in the task itself. -- This message was sent by Atlassian JIRA (v6.3.4#6332)