Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id D6B7E200D0B for ; Tue, 29 Aug 2017 01:09:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id CFD34165C49; Mon, 28 Aug 2017 23:09:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 21AB3165C47 for ; Tue, 29 Aug 2017 01:09:03 +0200 (CEST) Received: (qmail 60014 invoked by uid 500); 28 Aug 2017 23:09:02 -0000 Mailing-List: contact jira-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@kafka.apache.org Delivered-To: mailing list jira@kafka.apache.org Received: (qmail 60003 invoked by uid 99); 28 Aug 2017 23:09:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 28 Aug 2017 23:09:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id F2988183C0C for ; Mon, 28 Aug 2017 23:09:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id ySq8MXOSH9K2 for ; Mon, 28 Aug 2017 23:09:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 0FB2E5FE3A for ; Mon, 28 Aug 2017 23:09:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 636FDE04F7 for ; Mon, 28 Aug 2017 23:09:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 0B61D23F05 for ; Mon, 28 Aug 2017 23:09:00 +0000 (UTC) Date: Mon, 28 Aug 2017 23:09:00 +0000 (UTC) From: "Yingji Zhang (JIRA)" To: jira@kafka.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (KAFKA-5803) KafkaProducer should ignore invalid record from ProducerInterceptors MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 28 Aug 2017 23:09:05 -0000 Yingji Zhang created KAFKA-5803: ----------------------------------- Summary: KafkaProducer should ignore invalid record from ProducerInterceptors Key: KAFKA-5803 URL: https://issues.apache.org/jira/browse/KAFKA-5803 Project: Kafka Issue Type: Bug Components: clients Affects Versions: 0.11.0.0, 0.10.2.1, 0.10.2.0, 0.10.1.1 Reporter: Yingji Zhang Right now KafkaProducer does not verify the input record emitted from ProducerInterceptors. Thus if client sends a message to a non exist topic partition or ProducerInterceptors has a bug, or decides to early exit by returning a null, the doSend method of KafkaProducer will hung on metadata refreshment until maxBlockTimeMs has elapsed. This means production traffic will be blocked for each producer send call. In general, I feel like we should have a circuit break mechanism to allow early exit from producer. To do that, we can either (1) allow ProducerInterceptors to throw some type of CircuitBreaker exception or (2) let ProducerInterceptors return null as the protocol to indicate a disaster situation and make KafkaProducer's doSend early exit on null input. -- This message was sent by Atlassian JIRA (v6.4.14#64029)