From user-return-60176-archive-asf-public=cust-asf.ponee.io@cassandra.apache.org Tue Mar 6 09:34:52 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 9DCEE180652 for ; Tue, 6 Mar 2018 09:34:51 +0100 (CET) Received: (qmail 93978 invoked by uid 500); 6 Mar 2018 08:34:48 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 93967 invoked by uid 99); 6 Mar 2018 08:34:48 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Mar 2018 08:34:48 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 08199180374 for ; Tue, 6 Mar 2018 08:34:48 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=tink.se Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id I8lk1QmDDS7Q for ; Tue, 6 Mar 2018 08:34:46 +0000 (UTC) Received: from mail-qk0-f177.google.com (mail-qk0-f177.google.com [209.85.220.177]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 8EF255F588 for ; Tue, 6 Mar 2018 08:34:46 +0000 (UTC) Received: by mail-qk0-f177.google.com with SMTP id 130so23916567qkd.13 for ; Tue, 06 Mar 2018 00:34:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tink.se; s=tink; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=bqxIawsijBZKCK7HmTCN1XMm9GECm46IsXZ/oVBtOL0=; b=FwOp/M2uwfANVOG+tHMKtlZ87WtEwwuUgEsY1vBt95wFU2kMgZAvZ50Fa8hcnGzhLA pBcmF7dqFEEURqV89kwydyP0lyz53GCjwBgiDoISe4g9dnM/ynilHLpgUNdETz1wc793 sOI8izlqk+vSyDQUW5nwQ9FNo4qK/DmwdFdbk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=bqxIawsijBZKCK7HmTCN1XMm9GECm46IsXZ/oVBtOL0=; b=a/5Qo8KkOF7J4M0TFJBXJsEUXHR7Xw9G7LgUYKUTBaNimRASOzwxhbgxRpZpGZNhfS KGMm4xgjoMDBvUdfoIdVKmDR8MmYNrlFtPonlGqxQGM9YtDo8nny0vk7cHWrh8FGgshe NJclbnjYuPLgb1F1Rzt9t8dT3CSz96gip785PZB1cArVWKQfyC5xQ5VNq6kFrOyR7vxm erK85KTdTYKp7M6nu0fSFJCXsiHSb3BlxHZARLh7FEEbZn2XuhF0lb32ZYbbbVryj30e 5XbLw5LYS/23x7T7T2aWFBb1ZqiBOnfpaTidvd/CmHF7jaltmKGuTI9Lmg+O8KXrRLOA FCFA== X-Gm-Message-State: AElRT7GoTWOqIjDDaHoJ5sBIy6b8NjoLOofRRfOWSQckk59nAcw9tuvj 9pJOzppXu+ipg8JH1L4YhRRcyAK6DTLX9tOgqOGlsiEG X-Google-Smtp-Source: AG47ELvyPvG+O2IuSVmULH9me5LvnNbnBRvHspcJcRIXWsxt/NSaSzOP+SrQCjYE6TOjJKixXu62otPwJ9HQtiKIQvs= X-Received: by 10.55.164.85 with SMTP id n82mr26697320qke.342.1520325279999; Tue, 06 Mar 2018 00:34:39 -0800 (PST) MIME-Version: 1.0 References: <6CCD506D-8290-4D7B-9025-69D799C7B2D6@cisco.com> In-Reply-To: <6CCD506D-8290-4D7B-9025-69D799C7B2D6@cisco.com> From: Jens Rantil Date: Tue, 06 Mar 2018 08:34:29 +0000 Message-ID: Subject: Re: One time major deletion/purge vs periodic deletion To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary="94eb2c064b7041fdb50566ba50e0" --94eb2c064b7041fdb50566ba50e0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sounds like you are using Cassandra as a queue. It's an antibiotic pattern. What I would do would be to rely on TTL for removal of data and use the TWCS compaction strategy to handle removal and you just focus on insertion. On Tue, Mar 6, 2018, 07:39 Charulata Sharma (charshar) wrote: > Hi, > > > > Wanted the community=E2=80=99s feedback on deciding the schedule of= Archive > and Purge job. > > Is it better to Purge a large volume of data at regular intervals (like > run A&P jobs once in 3 months ) or purge smaller amounts more frequently > (run the job weekly??) > > > > Some estimates on the number of deletes performed would be=E2=80=A6upto 8= 0-90K > rows purged in 3 months vs 10K deletes every week ?? > > > > Thanks, > > Charu > > > --=20 Jens Rantil Backend Developer @ Tink Tink AB, Wallingatan 5, 111 60 Stockholm, Sweden For urgent matters you can reach me at +46-708-84 18 32. --94eb2c064b7041fdb50566ba50e0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sounds like you are using Cassandra as a queue. It's an antibiotic patt= ern. What I would do would be to rely on TTL for removal of data and use th= e TWCS compaction strategy to handle removal and you just focus on insertio= n.

On Tue, Mar 6, 2018, = 07:39 Charulata Sharma (charshar) <charshar@cisco.com> wrote:

Hi,

=C2=A0

=C2=A0=C2=A0=C2=A0= =C2=A0 =C2=A0Wanted the community=E2=80=99s feedback on deciding the schedu= le of Archive and Purge job. =C2=A0

Is it better to Pur= ge a large volume of data at regular intervals (like run A&P jobs once = in 3 months ) or purge smaller amounts more frequently (run the job weekly?= ?)

=C2=A0

Some estimates on t= he number of deletes performed would be=E2=80=A6upto 80-90K =C2=A0rows purg= ed in 3 months vs 10K deletes every week ??

=C2=A0

Thanks,

Charu=

=C2=A0

--

Jens Rantil
Backend Developer @ Tink

Tink AB, Wallingatan 5, 111 60 Stockholm, Sweden
F= or urgent matters you can reach me at +46-708-84 18 32.

--94eb2c064b7041fdb50566ba50e0--