From user-return-18257-archive-asf-public=cust-asf.ponee.io@flink.apache.org Mon Feb 19 16:47:00 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 72416180607 for ; Mon, 19 Feb 2018 16:46:59 +0100 (CET) Received: (qmail 98441 invoked by uid 500); 19 Feb 2018 15:46:53 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 98432 invoked by uid 99); 19 Feb 2018 15:46:53 -0000 Received: from mail-relay.apache.org (HELO mailrelay2-lw-us.apache.org) (207.244.88.137) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Feb 2018 15:46:53 +0000 Received: from mail-qk0-f179.google.com (mail-qk0-f179.google.com [209.85.220.179]) by mailrelay2-lw-us.apache.org (ASF Mail Server at mailrelay2-lw-us.apache.org) with ESMTPSA id 761F62DB for ; Mon, 19 Feb 2018 15:46:52 +0000 (UTC) Received: by mail-qk0-f179.google.com with SMTP id s198so12730097qke.5 for ; Mon, 19 Feb 2018 07:46:52 -0800 (PST) X-Gm-Message-State: APf1xPAmreeFI5cHsySz4JimksPSpX+/6UEHCDRYr4qwCRRWOJeMCwl+ Qz/EOaGOllcayCfyI/EF08q7LargDpOb4bSBn48= X-Google-Smtp-Source: AH8x226bYw0njs/rZ70JgaDJRMZ+SuDg4A7Yqfsl7iLyoniDfOkaFJZhOfGGX+lpFlyjckWpNjObEaupGxJm8OkTJoY= X-Received: by 10.55.19.129 with SMTP id 1mr24866012qkt.26.1519055212218; Mon, 19 Feb 2018 07:46:52 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.146.33 with HTTP; Mon, 19 Feb 2018 07:46:11 -0800 (PST) In-Reply-To: References: From: Till Rohrmann Date: Mon, 19 Feb 2018 16:46:11 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Question about the need of consumer groups from kafka To: Ricardo Dinis Cc: user Content-Type: multipart/alternative; boundary="001a1140173a519b740565929abd" --001a1140173a519b740565929abd Content-Type: text/plain; charset="UTF-8" Hi Ricardo, could you please give a bit more details what you mean with "not using its own mechanism"? Flink's Kafka connector uses the Kafka consumer and producer (to some extent) API to talk to Kafka. The consumer groups are a central concept of Kafka and as such, the Flink Kafka connector has to know it in order to consume from the right group. Creating a fat-jar will make sure that the jar includes additional dependencies such as the flink-kafka-connector which is not present on the Flink cluster. Cheers, Till On Fri, Feb 16, 2018 at 2:05 PM, Ricardo Dinis wrote: > Hi, > > If flink can manage kafka without using its own's mechanism (kafka > consumers) it still need the consumer group defined from kafka? > > And why the examples is needed to use the plugin for shadding creating a > fat-jar? > > Thanks > --001a1140173a519b740565929abd Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Ricardo,

could you please give a bit= more details what you mean with "not using its own mechanism"? F= link's Kafka connector uses the Kafka consumer and producer (to some ex= tent) API to talk to Kafka. The consumer groups are a central concept of Ka= fka and as such, the Flink Kafka connector has to know it in order to consu= me from the right group.

Creating a fat-jar will mak= e sure that the jar includes additional dependencies such as the flink-kafk= a-connector which is not present on the Flink cluster.

=
Cheers,
Till
On Fri, Feb 16, 2018 at 2:05 PM, Ricardo Dinis = <dinsird@gmail.com> wrote:
Hi,

If flink can manage kafka without = using its own's mechanism (kafka consumers) it still need the consumer = group defined from kafka?

And why the examples is = needed to use the plugin for shadding creating a fat-jar?

Thanks

--001a1140173a519b740565929abd--