Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 31795200BD8 for ; Wed, 7 Dec 2016 17:45:40 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 30046160B0C; Wed, 7 Dec 2016 16:45:40 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 55B54160AF9 for ; Wed, 7 Dec 2016 17:45:39 +0100 (CET) Received: (qmail 97160 invoked by uid 500); 7 Dec 2016 16:45:37 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 97150 invoked by uid 99); 7 Dec 2016 16:45:37 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Dec 2016 16:45:37 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 803FE181A6E for ; Wed, 7 Dec 2016 16:45:37 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=jaumo.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id sTPFvp3IYU21 for ; Wed, 7 Dec 2016 16:45:35 +0000 (UTC) Received: from mail-ua0-f181.google.com (mail-ua0-f181.google.com [209.85.217.181]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 4613F5F4EC for ; Wed, 7 Dec 2016 16:45:35 +0000 (UTC) Received: by mail-ua0-f181.google.com with SMTP id b35so420228076uaa.3 for ; Wed, 07 Dec 2016 08:45:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jaumo.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=uxu16XeU97+Q8M0HINA76KilxznWDXyCdRilKc9AEqI=; b=nLD3f1XUGP5fi30KJ02NDXHypu9tXRMUTf3qRClftnt1rKZLOZXiXWXEZc5K04h1zv ouTVhFhadkSc/pQUK0tbsrkFve1oDN2MO0VVGNla+fLz2j6DOiyRLuPtzpPF9z3l2gnM l+SpvZc7vFsBE9QuqrsX8rPWi7dUBu/Rvy0+w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=uxu16XeU97+Q8M0HINA76KilxznWDXyCdRilKc9AEqI=; b=B9kBClipGEYXZhuv9n6I+kYC+yiKBzXClexs6C9mS1ngifu1bzKa8VROuJOhCDrIu0 i1sgG7FnKRBLxzUXAV2MMye2tBnSHJwjgpdgwrDIusi5ZjjM22VaoeeomXguZGBz7GtQ enIV1wMT7IDw2GG0xY/06Dicplw3ZDrygzm2XrnyWWpNVk21gRFw5TqCgrUVQci+33ta KtIbY+mlVTiEB9wx3pmyVf16bugQjx/fKMU2s+oLJbyNgm7kawgFr/5PnYett66jwZpu eFnGuVY3fQC7y9Ks36Reh+iDv53IVzpq1IH2sAvxu5Cux8xGDmWhT37lXCU10Rf6IIfw vi8Q== X-Gm-Message-State: AKaTC00prJhbCKuLrqSbjTxD0h7e9R9hh8QqCJdcXpyUyktqW59L6MZJt9oLqnGN4iR8Ve+KvD04EMK58HVZeQ7i X-Received: by 10.176.16.78 with SMTP id g14mr42621879uab.54.1481129134815; Wed, 07 Dec 2016 08:45:34 -0800 (PST) MIME-Version: 1.0 Received: by 10.176.81.144 with HTTP; Wed, 7 Dec 2016 08:45:34 -0800 (PST) Received: by 10.176.81.144 with HTTP; Wed, 7 Dec 2016 08:45:34 -0800 (PST) In-Reply-To: References: From: Benjamin Roth Date: Wed, 7 Dec 2016 17:45:34 +0100 Message-ID: Subject: Re: Batch size warnings To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=f403045e2f12f290810543143f66 archived-at: Wed, 07 Dec 2016 16:45:40 -0000 --f403045e2f12f290810543143f66 Content-Type: text/plain; charset=UTF-8 I meant the mv thing Am 07.12.2016 17:27 schrieb "Voytek Jarnot" : > Sure, about which part? > > default batch size warning is 5kb > I've increased it to 30kb, and will need to increase to 40kb (8x default > setting) to avoid WARN log messages about batch sizes. I do realize it's > just a WARNing, but may as well avoid those if I can configure it out. > That said, having to increase it so substantially (and we're only dealing > with 5 tables) is making me wonder if I'm not taking the correct approach > in terms of using batches to guarantee atomicity. > > On Wed, Dec 7, 2016 at 10:13 AM, Benjamin Roth > wrote: > >> Could you please be more specific? >> >> Am 07.12.2016 17:10 schrieb "Voytek Jarnot" : >> >>> Should've mentioned - running 3.9. Also - please do not recommend MVs: >>> I tried, they're broken, we punted. >>> >>> On Wed, Dec 7, 2016 at 10:06 AM, Voytek Jarnot >>> wrote: >>> >>>> The low default value for batch_size_warn_threshold_in_kb is making me >>>> wonder if I'm perhaps approaching the problem of atomicity in a non-ideal >>>> fashion. >>>> >>>> With one data set duplicated/denormalized into 5 tables to support >>>> queries, we use batches to ensure inserts make it to all or 0 tables. This >>>> works fine, but I've had to bump the warn threshold and fail threshold >>>> substantially (8x higher for the warn threshold). This - in turn - makes >>>> me wonder, with a default setting so low, if I'm not solving this problem >>>> in the canonical/standard way. >>>> >>>> Mostly just looking for confirmation that we're not unintentionally >>>> doing something weird... >>>> >>> >>> > --f403045e2f12f290810543143f66 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

I meant the mv thing


Am 07.12.2016 17:= 27 schrieb "Voytek Jarnot" <voytek.jarnot@gmail.com>:
Sure, about which part?

default batch size warning is 5kb
I've increased it to= 30kb, and will need to increase to 40kb (8x default setting) to avoid WARN= log messages about batch sizes.=C2=A0 I do realize it's just a WARNing= , but may as well avoid those if I can configure it out.=C2=A0 That said, h= aving to increase it so substantially (and we're only dealing with 5 ta= bles) is making me wonder if I'm not taking the correct approach in ter= ms of using batches to guarantee atomicity.

On Wed, Dec 7, 2016 at 10:13 AM, Benj= amin Roth <benjamin.roth@jaumo.com> wrote:

Could you please be more specific?


Am 07.12.2016 17:= 10 schrieb "Voytek Jarnot" <voytek.jarnot@gmail.com>:
Should've ment= ioned - running 3.9.=C2=A0 Also - please do not recommend MVs: I tried, the= y're broken, we punted.

On Wed, Dec 7, 2016 at 10:06 AM, Voytek Jarnot <v= oytek.jarnot@gmail.com> wrote:
The low default value for=C2=A0batch_size_warn_thresho= ld_in_kb is making me wonder if I'm perhaps approaching the proble= m of atomicity in a non-ideal fashion.

With one data set= duplicated/denormalized into 5 tables to support queries, we use batches t= o ensure inserts make it to all or 0 tables.=C2=A0 This works fine, but I&#= 39;ve had to bump the warn threshold and fail threshold substantially (8x h= igher for the warn threshold).=C2=A0 This - in turn - makes me wonder, with= a default setting so low, if I'm not solving this problem in the canon= ical/standard way.

Mostly just looking for confirm= ation that we're not unintentionally doing something weird...


--f403045e2f12f290810543143f66--