From jira-return-60333-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Tue Dec 29 20:59:32 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-he-de.apache.org (mxout1-he-de.apache.org [95.216.194.37]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 85F00180626 for ; Tue, 29 Dec 2020 21:59:32 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-he-de.apache.org (ASF Mail Server at mxout1-he-de.apache.org) with SMTP id D1F6E654D2 for ; Tue, 29 Dec 2020 20:59:31 +0000 (UTC) Received: (qmail 76402 invoked by uid 500); 29 Dec 2020 20:59:31 -0000 Mailing-List: contact jira-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@kafka.apache.org Delivered-To: mailing list jira@kafka.apache.org Received: (qmail 76388 invoked by uid 99); 29 Dec 2020 20:59:31 -0000 Received: from mailrelay1-he-de.apache.org (HELO mailrelay1-he-de.apache.org) (116.203.21.61) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Dec 2020 20:59:31 +0000 Received: from jira2-he-de.apache.org (jira2-he-de.apache.org [168.119.33.54]) by mailrelay1-he-de.apache.org (ASF Mail Server at mailrelay1-he-de.apache.org) with ESMTPS id 4D7403E83E for ; Tue, 29 Dec 2020 20:59:30 +0000 (UTC) Received: from jira2-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira2-he-de.apache.org (ASF Mail Server at jira2-he-de.apache.org) with ESMTP id 300E4C80245 for ; Tue, 29 Dec 2020 20:59:00 +0000 (UTC) Date: Tue, 29 Dec 2020 20:59:00 +0000 (UTC) From: "Matthias J. Sax (Jira)" To: jira@kafka.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (KAFKA-10892) Add Topology#connectSourceStoreAndTopic as a public method MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/KAFKA-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-10892: ------------------------------------ Issue Type: New Feature (was: Improvement) > Add Topology#connectSourceStoreAndTopic as a public method > ---------------------------------------------------------- > > Key: KAFKA-10892 > URL: https://issues.apache.org/jira/browse/KAFKA-10892 > Project: Kafka > Issue Type: New Feature > Components: streams > Affects Versions: 2.7.0 > Reporter: Tomohiro Hashidate > Priority: Major > Labels: need-kip > > I want Topology#connectSourceStoreAndTopic. > Because I want to use a topic as a source topic directly without a redundant changelog topic for not only KeyValueStore but also WindowStore. > This issue is similar to [KAFKA-6840], but is a suggestion for a more general approach > {code:java} > public synchronized Topology connectSourceStoreAndTopic(final String sourceStoreName, > final String topic) { > internalTopologyBuilder.connectSourceStoreAndTopic(sourceStoreName, topic); > return this; > } > {code} > h3. Background > I want to use a topic as a source topic for WindowStore because using WindowStore is suitable for the feature that I'm implementing. > The records stored in the topic are aggregated with a time window by another application. The size of the topic is over 10TB. > I want to use the topic as a source topic for WindowStore directly. > But, I cannot do so on the current interface. > I need a redundant topic only for storing the records into WindowStore. > If this API is public, I can use topics incoming from other applications (not only Kafka Streams applications) as source topics for any StateStore implementation without redundant changelog topics. > Of course, I need to implement a processor for storing incoming records in such a case. > But I think it's not difficult. > Please consider this. -- This message was sent by Atlassian Jira (v8.3.4#803005)