Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Wed, 2 Dec 2015 08:33:10 +0000 (UTC)
From: "Pawel Matras (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12917208.1448964579000.242830.1449045190948@Atlassian.JIRA>
In-Reply-To: <JIRA.12917208.1448964579000@Atlassian.JIRA>
References: <JIRA.12917208.1448964579000@Atlassian.JIRA>
 <JIRA.12917208.1448964579025@arcas>
Subject: [jira] [Commented] (CASSANDRA-10792) Make warning level for count
 of partitions in unlogged BatchStatement configurable.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/CASSANDRA-10792?page=3Dcom.atla=
ssian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=
=3D15035464#comment-15035464 ]=20

Pawel Matras commented on CASSANDRA-10792:
------------------------------------------

Sorry for creating confusion. Perhaps this comes from too long explanation =
or perhaps because I would like to give my vote for unlogged batches as the=
y do perform better in some deployment scenarios (as I tried to explain abo=
ve). Actually, I am a little bit afraid that they are on their way out of C=
assandra as you recommend against them and warn users that they should do s=
omething else instead.

{quote}
It is my understanding that while we recommend against it, submitting batch=
es for multiple partitions is currently possible. Is this not the case?
{quote}
Yes, it is possible. Creating of unlogged batches for multiple partitions w=
orks. Unfortunately in newer versions of Cassandra they produce lots of war=
nings in the server logs and it is quite hard to turn these warnings off. S=
o my  improvement proposal suggests to introduce configuration parameter in=
 cassandra.yaml e.g.
{noformat}
unlogged_batch_partitions_warn_threshold: 1
{noformat}
similar to existing {{batch_size_warn_threshold_in_kb}}. This new parameter=
 will then be used in BatchStatement.verifyBatchType instead of todays hard=
coded 1.

{quote}
What is the CQL syntax of the statement you are trying, and the schema of t=
he table you are using?
{quote}
I am using DataStax java driver. Batch statement itself is created programm=
atically {{new BatchStatement( BatchStatement.Type.UNLOGGED )}}. Then  inse=
rts are added to it by binding PreparedStatement created from CQL. Batch is=
 executed asynchronously. Table is similar to (sorry, I am not allowed to p=
rovide details):
{noformat}
create table MY_TABLE (
   C1 varchar,
   C2 varchar,
   C3 varchar,
   C4 blob,
   primary key( (C1, C2), C3 )
);
{noformat}

In hope of clarification, best regards

Pawe=C5=82

> Make warning level for count of partitions in unlogged BatchStatement con=
figurable.
> -------------------------------------------------------------------------=
----------
>
>                 Key: CASSANDRA-10792
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1079=
2
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Coordination
>         Environment: CentOS,  JDK 1.7, cluster of servers with 32 CPU eac=
h.
>            Reporter: Pawel Matras
>            Priority: Minor
>             Fix For: 3.2
>
>
> Currently mutations for only one partition are allowed in BatchStatement.=
 This amount is hard coded and not configurable as e.g. batch size.
> General suggestion looks to be to consider multi partition unlogged batch=
 statements as distributed anti pattern. As cure async inserts are proposed=
. Proposal might be OK if one consider only cassandra side of the system.
> If a complete system have to share the same hardware things does not look=
 so obvious any more. When cassandra shares the same hardware with other al=
so clustered components lots of small async inserts become another well kno=
wn distributed anti pattern.=20
> In our case changing from unlogged batches to async inserts destabilize t=
he system as async inserts require amounts of interrupts per second and con=
text switches per second beyond what hardware can handle. With replica awar=
e unlogged batches these parameters are at 40% of hardware limits and the s=
ystem as a whole runs stable without visible hotspots (cassandra metrics lo=
oks almost the same on all nodes).
> Unfortunately latest versions (we switched from 2.0.6 to 2.2.1) of cassan=
dra log lots of warnings of type "Unlogged batch covering NN partitions det=
ected against table XXX...". I found two workarounds to avoid this warning.=
 The one is to use logged batches. But they generate 20% more interrupts an=
d context switches and 400% more network load. The other way is more a hack=
 and uses filtering features of logging framework and suppresses the warnin=
g just before it get logged.
> So my suggestion is to allow users of cassandra to decide by configuratio=
n if they consider unlogged batches antipattern or not. This is partially d=
one with the size of the batch, where size is configurable and not hardcode=
d to 5 kB.
> It would be fine to stay consistent on this and let the user configure mu=
tations for how many partitions to allow.
> Of course there are several other solutions possible here, probably more =
costly. E.g warning could be produced only if batch is not token/replica aw=
are.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)