From user-return-63997-archive-asf-public=cust-asf.ponee.io@cassandra.apache.org Fri Jun 7 22:32:28 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id A05A718067E for ; Sat, 8 Jun 2019 00:32:28 +0200 (CEST) Received: (qmail 62765 invoked by uid 500); 7 Jun 2019 22:32:24 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 62752 invoked by uid 99); 7 Jun 2019 22:32:24 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Jun 2019 22:32:24 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 8AF96C079B for ; Fri, 7 Jun 2019 22:32:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.801 X-Spam-Level: * X-Spam-Status: No, score=1.801 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=smartthings.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id A2QRy0AtNKLh for ; Fri, 7 Jun 2019 22:32:21 +0000 (UTC) Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 5AC4F5F175 for ; Fri, 7 Jun 2019 22:32:21 +0000 (UTC) Received: by mail-ed1-f53.google.com with SMTP id z25so5067813edq.9 for ; Fri, 07 Jun 2019 15:32:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=SZHTC8PQEIWHpl+WWczAxHTZ2oMMGTEdw5uxvJsM5ho=; b=RXU0xOLWNVmUinhoiTS8xrFJiRdHKbt8V9nhyCi3AD5E9593xg5CqLtx7ddLyQrHwG KJmvw6QxxYge/feyot9wzBYCKvn/PLkRUxjsVY10nxcSA/llu9tYSnS0FmLdOtBKPG2z TFBxmfGX3mq51NQsGTSeXakviQJOPoeljUW72cObZLTDuNh3dh0z0UK5Gct7JL8dbzUD W898buPGXW/LnUtggv8goa4hxCttIlhxGwuh/zAABMi+DW32bJiO26y5EW8dBngIXwjo CNmNN70e6LHh7k4L/YB9Lr4yQgwFZOG/mvtLljFIcmfC6kxrWTpWo+kFuzo+dI3AvW6e wEVA== X-Gm-Message-State: APjAAAWiW1W0oxIgKlboPshUPFPEUNKlnDcNlDnZpe0jEqC8+PpeG++7 3yfpjL7x+lLM8fxk6Q/koVhR8Fry2DLrcZSVMT8fVA3X X-Google-Smtp-Source: APXvYqyoZOs0ydHnAIMYK5gon/lBV8KaQkRAHRuCZCPeLRnGRI75buP6g3lfWn2ik2pCGrqKQcONTOF6kDUDKSTTLZs= X-Received: by 2002:a17:906:b6cb:: with SMTP id ec11mr48887613ejb.215.1559946739963; Fri, 07 Jun 2019 15:32:19 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Carl Mueller Date: Fri, 7 Jun 2019 17:32:09 -0500 Message-ID: Subject: Re: schema for testing that has a lot of edge cases To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary="0000000000004df862058ac367e1" --0000000000004df862058ac367e1 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks. On Mon, May 27, 2019 at 12:24 PM Alain RODRIGUEZ wrote= : > Hello Carl, > > What you try to do sounds like a good match with one of the tool we > open-sourced and actively maintain: > https://github.com/thelastpickle/tlp-stress. > > TLP Stress allows you to use defined profiles (see > https://github.com/thelastpickle/tlp-stress/tree/master/src/main/kotlin/c= om/thelastpickle/tlpstress/profiles) > or create your own profiles and/or schemas. Contributions are welcome. Yo= u > can tune workloads, the read/write ratio, the number of distinct > partitions, number of operations to run... > > You might need multiple client to maximize the throughput, depending on > instances in use and your own testing goals. > > version specific stuff to 2.1, 2.2, 3.x, 4.x > > > In case that might be of some use as well, we like to use it combined wit= h > another of our tools: TLP Cluster ( > https://github.com/thelastpickle/tlp-cluster). We can the easily create > and destroy Cassandra environments (on AWS) including Cassandra servers, > client and monitoring (Prometheus). > > You can have a look anyway, I think both projects might be of interest to > reach your goal. > > C*heers, > ----------------------- > Alain Rodriguez - alain@thelastpickle.com > France / Spain > > The Last Pickle - Apache Cassandra Consulting > http://www.thelastpickle.com > > > Le jeu. 23 mai 2019 =C3=A0 21:25, Carl Mueller > a =C3=A9crit : > >> Does anyone have any schema / schema generation that can be used for >> general testing that has lots of complicated aspects and data? >> >> For example, it has a bunch of different rk/ck variations, column data >> types, altered /added columns and data (which can impact sstables and >> compaction), >> >> Mischeivous data to prepopulate (such as >> https://github.com/minimaxir/big-list-of-naughty-strings for strings, >> ugly keys in maps, semi-evil column names) of sufficient size to get on >> most nodes of a 3-5 node cluster >> >> superwide rows >> large key values >> >> version specific stuff to 2.1, 2.2, 3.x, 4.x >> >> I'd be happy to centralize this in a github if this doesn't exist >> anywhere yet >> >> >> --0000000000004df862058ac367e1 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks.=C2=A0

On Mon, May 27, 2019 at 12:24 PM Alain RODRIG= UEZ <arodrime@gmail.com> wr= ote:
<= div dir=3D"ltr">Hello Carl,

What you try to do sounds li= ke a good match with one of the tool we open-sourced and actively maintain:= = https://github.com/thelastpickle/tlp-stress.



version specific stuff to 2.1, 2.2, 3.x, 4.x
=C2=A0
In case that might be of some use as well, we li= ke to use it combined with another of our tools: TLP Cluster (https://githu= b.com/thelastpickle/tlp-cluster). We can the easily create and destroy = Cassandra environments (on AWS) including Cassandra servers, client and mon= itoring (Prometheus).

You can have a look anyway, = I think both projects might be of interest to reach your goal.
C*heers,
-----------------------
Alai= n Rodriguez - = alain@thelastpickle.com
France / Spain

The Last Pickle - Apache Cassandra Consulting


Le=C2=A0jeu. 23 ma= i 2019 =C3=A0=C2=A021:25, Carl Mueller <carl.mueller@smartthings.com.inv= alid> a =C3=A9crit=C2=A0:
Does anyone have any schema / schema generati= on that can be used for general testing that has lots of complicated aspect= s and data?

For example, it has a bunch of different rk/ck variation= s, column data types, altered /added columns and data (which can impact sst= ables and compaction),=C2=A0

Mischeivous data to prepopulate (such a= s=C2=A0https://github.com/minimaxir/big-list-of-naughty-strings= =C2=A0for strings, ugly keys in maps, semi-evil column names) of suffic= ient size to get on most nodes of a 3-5 node cluster

sup= erwide rows
large key values

version specific s= tuff to 2.1, 2.2, 3.x, 4.x

I'd be happy to cen= tralize this in a github if this doesn't exist anywhere yet
<= br>
--0000000000004df862058ac367e1--