From dev-return-107443-archive-asf-public=cust-asf.ponee.io@kafka.apache.org Thu Sep 12 17:03:13 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5D7BB180651 for ; Thu, 12 Sep 2019 19:03:13 +0200 (CEST) Received: (qmail 67802 invoked by uid 500); 12 Sep 2019 17:03:12 -0000 Mailing-List: contact dev-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@kafka.apache.org Delivered-To: mailing list dev@kafka.apache.org Received: (qmail 67791 invoked by uid 99); 12 Sep 2019 17:03:11 -0000 Received: from ui-eu-02.ponee.io (HELO localhost) (116.202.110.96) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Sep 2019 17:03:11 +0000 Content-Type: text/plain; charset=utf-8 X-Mailer: LuaSocket 3.0-rc1 In-Reply-To: <0a147216-c0bc-4334-ac81-00812472f912@www.fastmail.com> x-ponymail-agent: PonyMail Composer/0.2 x-ponymail-sender: ac90dfeb5a56f50a058ba0d008644af4ae5565f0 Message-ID: Date: Thu, 12 Sep 2019 17:03:10 -0000 References: <0a147216-c0bc-4334-ac81-00812472f912@www.fastmail.com> Subject: Re: _consumer_offsets is becoming rather big. How to purge? From: Ash G To: MIME-Version: 1.0 Thanks Colin, that is the reason. On 2019/09/11 17:16:46, "Colin McCabe" wrote: > Hi Ash, > > At first guess, you probably had a problem with your log cleaner thread, which resulted in the offsets log not being cleaned. Check if that thread is running. > > best, > Colin > > On Wed, Sep 11, 2019, at 09:52, Ash G wrote: > > Bump, no reply, > > > > It seems this condition was missed by devs when this feature was > > designed and implemented. > > > > On 2019/09/06 14:45:47, Ash G wrote: > > > > > > _consumer_offsets is becoming rather big > 1 TB. Is there a way to purge dead/inactive consumer id rows from it? > > > I am assuming dead/inactive consumer id rows is the reason. could there be another reason? > > > > > > [reposting from user list] > > > > > >