Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4F0B918841 for ; Wed, 5 Aug 2015 03:25:52 +0000 (UTC) Received: (qmail 53104 invoked by uid 500); 5 Aug 2015 03:25:49 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 53066 invoked by uid 500); 5 Aug 2015 03:25:49 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 53056 invoked by uid 99); 5 Aug 2015 03:25:49 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Aug 2015 03:25:49 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id BC5EF1A97A0 for ; Wed, 5 Aug 2015 03:25:48 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.9 X-Spam-Level: ** X-Spam-Status: No, score=2.9 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id ycK8mFkXNNgj for ; Wed, 5 Aug 2015 03:25:38 +0000 (UTC) Received: from mail-ob0-f174.google.com (mail-ob0-f174.google.com [209.85.214.174]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 7EEE542B0A for ; Wed, 5 Aug 2015 03:25:38 +0000 (UTC) Received: by obnw1 with SMTP id w1so22257899obn.3 for ; Tue, 04 Aug 2015 20:24:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=XXja1aEGcgWXmQ1EH12icz23oc1eIh4blE0vcaIx9uI=; b=Y1+HTrlmDXE2kaaxNowB2LptFfJxfMvpoMOh9a2yRscXtId5L2tTZT+mu0aqeh6MWt JR8QASUVPs9/DpXitZgka1nAlzSTyJCO4J0Sh7P9oWT8b9geH7nIxyMyEFFdq9alWUtx zJjqu+BkX9p7rDqzeiTy/uZ0eXWiwSYSY/2gIEo3HWYluImJGxvAmhUyT94gJU38J9wA STA9qUl1bmfjSh9KcSzQf63+cGRgJbdehVVw5WA530vIu78Uyz0vfD5Lj0KVQEFYrH9X XJwHWviOKf3lalFI72fCLds+l7oMtVfcTN2NfrcDxQjS5L4JnlzcEtttij4PpAk6eVxE AJQQ== MIME-Version: 1.0 X-Received: by 10.182.86.9 with SMTP id l9mr6150264obz.61.1438745041799; Tue, 04 Aug 2015 20:24:01 -0700 (PDT) Received: by 10.202.1.194 with HTTP; Tue, 4 Aug 2015 20:24:01 -0700 (PDT) In-Reply-To: References: <7C88CB4D-29D8-495A-93E0-BF82716392E9@ecyrd.com> Date: Tue, 4 Aug 2015 20:24:01 -0700 Message-ID: Subject: Re: Cassandra Data Stax java driver & Snappy Compression library From: Sachin Nikam To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=089e014954c8238e49051c87ef1d --089e014954c8238e49051c87ef1d Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Janne, Thanks for continuing to take the time to answer my queries. We noticed that write latency (tp99) from Services S1 and S2 is 50% of the write latency (tp99) for Service S3. I also noticed that S1 and S2, which also use astyanax client library also have compress-lzf.jar on their class path. Although the table is defined to use Snappy Compression. Is this compression library or some other transitive dependency pulled in by Astyanax enabling compression of the payload i.e. sent over the wire and account for the difference in tp99? Regards Sachin On Mon, Aug 3, 2015 at 12:14 AM, Janne Jalkanen wrote: > > Correct. Note that you may lose some performance this way though; in a > typical case saving bandwidth by increasing CPU usage is good. However, i= t > always depends on your usecase and whether you=E2=80=99re running your cl= uster to > the max. It=E2=80=99s a good, low-hanging optimization to keep in mind th= ough for > production environments, if you choose not to enable compression now. > > /Janne > > On 3 Aug 2015, at 08:40, Sachin Nikam wrote: > > Thanks Janne... > To clarify, Service S3 should not run in to any issues and I may choose t= o > not fix the issue? > Regards > Sachin > > On Sat, Aug 1, 2015 at 11:50 PM, Janne Jalkanen > wrote: > >> No, this just tells that your client (S3 using Datastax driver) cannot >> communicate to the Cassandra cluster using a compressed protocol, since = the >> necessary libraries are missing on the client side. Servers will still >> compress the data they receive when they write it to disk. >> >> In other words >> >> Client <- [uncompressed data] -> Server <- [compressed data] -> Disk. >> >> To fix, make sure that the Snappy libraries are in the classpath of your >> S3 service application. As always, there=E2=80=99s no guarantee that th= is improves >> your performance, since if your app is already CPU-heavy, the extra CPU >> overhead of compression *may* be a problem. So measure :-) >> >> /Janne >> >> On 02 Aug 2015, at 02:17 , Sachin Nikam wrote: >> >> I am currently running a Cassandra 1.2 cluster. This cluster has 2 table= s >> i.e. >> TableA and TableB. >> >> TableA is read and written to by Services S1 and S2 which use Astyanax >> client library. >> >> TableB is read and written by Service S3 which uses the datastax java >> driver 2.1. S3 also reads data from TableA. >> >> Both TableA and TableB are defined on the Cassandra nodes to use >> SnappyCompressor. >> >> On start-up service, Service S3 throws the following WARNing messages. >> The service is able to continue doing its normal operation thereafter >> >> ************** >> [main] WARN loggerClass=3Dcom.datastax.driver.core.FrameCompressor;Canno= t >> find Snappy class, you should make sure the Snappy library is in the >> classpath if you intend to use it. Snappy compression will not be >> available for the protocol. >> *********** >> >> >> My questions are as follows-- >> #1. Does the compression happen on the cassandra client side or within >> cassandra server side itself? >> #2. Does Service S3 need to pull in additional dependencies for Snappy >> Compressions as mentioned here -- >> >> http://stackoverflow.com/questions/21784149/getting-cassandra-connection= -error >> #3. What happens without this additional library not being present on >> class path of Service S3. Any data that S3 writes to TableB will not be >> compressed? >> Regards >> Sachin >> >> >> > > --089e014954c8238e49051c87ef1d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Janne,
Thanks for continuing to take the time to answe= r my queries. We noticed that write latency (tp99) from Services S1 and S2 = is 50% of the write latency (tp99) for Service S3. I also noticed that S1 a= nd S2, which also use astyanax client library also have compress-lzf.jar on= their class path. Although the table is defined to use Snappy Compression.= Is this compression library or some other transitive dependency pulled in = by Astyanax enabling compression of the payload i.e. sent over the wire and= account for the difference in tp99?
Regards
Sachin

On Mon, A= ug 3, 2015 at 12:14 AM, Janne Jalkanen <janne.jalkanen@ecyrd.com> wrote:

Correct. Note that you may lose some per= formance this way though; in a typical case saving bandwidth by increasing = CPU usage is good. However, it always depends on your usecase and whether y= ou=E2=80=99re running your cluster to the max. It=E2=80=99s a good, low-han= ging optimization to keep in mind though for production environments, if yo= u choose not to enable compression now.

/Janne


Thanks Janne.= ..
To clarify, Service S3 should not run in to any issues and I may cho= ose to not fix the issue?
Regards
Sachin

On Sat, Aug 1, 2015 = at 11:50 PM, Janne Jalkanen <Janne.Jalkanen@ecyrd.com> wrote:
No, this just tells that your client (S3 using Datastax driver) ca= nnot communicate to the Cassandra cluster using a compressed protocol, sinc= e the necessary libraries are missing on the client side.=C2=A0 Servers wil= l still compress the data they receive when they write it to disk.

In other words

Client =C2=A0<-= [uncompressed data] -> Server <- [compressed data] -> Disk.=C2=A0=

To fix, make sure that the Snappy libraries are i= n the classpath of your S3 service application.=C2=A0 As always, there=E2= =80=99s no guarantee that this improves your performance, since if your app= is already CPU-heavy, the extra CPU overhead of compression *may* be a pro= blem.=C2=A0 So measure :-)

/Janne

On 02 Aug 2015, at 02:17 , Sachin Nikam <sknikam@gmail.com> wrote:

<= div>
I am currently running a Cassandra 1.2 cluster. This c= luster has 2 tables i.e.
TableA and TableB.

Ta= bleA is read and written to by Services S1 and S2 which use Astyanax client= library.

TableB is read and written by Service S3= which uses the datastax java driver 2.1. S3 also reads data from TableA.

Both TableA and Ta= bleB are defined on the Cassandra nodes to use SnappyCompressor.
=

On start-up service, Service S3 throws the follow= ing WARNing messages. The service is able to continue doing its normal oper= ation thereafter

**************
[main] WARN loggerClass=3Dcom.datastax.driver.= core.FrameCompressor;Cannot find = Snappy class, you = should make sure the= S= nappy library is <= span style=3D"color:rgb(51,51,51);font-family:'Droid Sans Mono',Con= solas,Monaco,'Courier New',Courier,monospace;font-size:12px;line-he= ight:16px;white-space:pre-wrap">in = the classpath if you= intend to use it. Snappy compre= ssion will not be <= /span>available for the pro= tocol.
***********


<= /div>
My questions are as follows--
#1. Does the compression ha= ppen on the cassandra client side or within cassandra server side itself?
#2. Does Service S3 need t= o pull in additional dependencies for Snappy Compressions as mentioned here= --
#3. What happens without this additional library not being present= on class path of Service S3. Any data that S3 writes to TableB will not be= compressed?
Regards
Sachin




--089e014954c8238e49051c87ef1d--