Return-Path: 
 <user-return-41326-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org>
X-Original-To: apmail-cassandra-user-archive@www.apache.org
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id A9A0911847
	for <apmail-cassandra-user-archive@www.apache.org>;
 Thu, 19 Jun 2014 20:40:11 +0000 (UTC)
Received: (qmail 95722 invoked by uid 500); 19 Jun 2014 20:40:08 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 95678 invoked by uid 500); 19 Jun 2014 20:40:08 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@cassandra.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@cassandra.apache.org>
List-Post: <mailto:user@cassandra.apache.org>
List-Id: <user.cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 95668 invoked by uid 99); 19 Jun 2014 20:40:08 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Jun 2014 20:40:08 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of rbradberry@gmail.com designates
 209.85.220.46 as permitted sender)
Received: from [209.85.220.46] (HELO mail-pa0-f46.google.com) (209.85.220.46)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Jun 2014 20:40:05 +0000
Received: by mail-pa0-f46.google.com with SMTP id eu11so2267388pac.19
        for <user@cassandra.apache.org>; Thu, 19 Jun 2014 13:39:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=date:from:to:message-id:in-reply-to:references:subject:mime-version
         :content-type;
        bh=xCFfk5X/r8P7tnoienITalVLavsG+N5H8OubT2IWMVc=;
        b=uQBdwnq9A3ELrlE9D8Ez+FbDbFNiOpbkEyqpzU/UuV0FlutjRDKNvHKHFxBbfk0CH7
         5/MVVjUjSH2zsMvAjFDjD/p7p7jVb3Qvzf/qe9ka7QCGBJk3wDVhNuzjau9dv225IxD5
         8m5ROBe1LW9Kj2GWq4FE56HjvrV/8FvkjQiRijfUUz+oFPX4JuxS8X67PuybwIoe6Zee
         aCCvA9Hw2X9FIsvXt08UZq+zWclnrgjU/sSZUD2ej2lWawuTIjFO4KCtS7RNic5D0qVe
         V0yuyxW2XgH7YWT+JNKoQZ8Wvp54nPhkc9A+Iw5bpgE3V06cDJeZA7yKhupE0Jpn3aXC
         8cvA==
X-Received: by 10.66.161.69 with SMTP id xq5mr8619670pab.62.1403210380152;
        Thu, 19 Jun 2014 13:39:40 -0700 (PDT)
Received: from Russells-iMac.local (rrcs-24-136-119-106.nyc.biz.rr.com.
 [24.136.119.106])
        by mx.google.com with ESMTPSA id
 xs2sm31905174pab.0.2014.06.19.13.39.37
        for <multiple recipients>
        (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
        Thu, 19 Jun 2014 13:39:39 -0700 (PDT)
Date: Thu, 19 Jun 2014 16:39:36 -0400
From: Russell Bradberry <rbradberry@gmail.com>
To: Nate McCall <nate@thelastpickle.com>, user@cassandra.apache.org
Message-ID: <etPan.53a34a88.74b0dc51.3cd@Russells-iMac.local>
In-Reply-To: 
 <CAKmMYa8KvMFt-Kp2=boFBcWCWsAChJJt4F3UHkugL4xJNLanWw@mail.gmail.com>
References: 
 <CA+VSrLp10hs9wVx9izaHczqZBVEpdYQ5Zbn7nNtK0hGuWPJzEg@mail.gmail.com>
 <CA+VSrLrxrGGPWv3nySDJxnF3apbpq7txEzxAfHS=4P=sV98FoA@mail.gmail.com>
 <CAOWvczKc=MOZQwYsc0zYdc2amQRjZcbPHMR40CUTwaumHSj3XA@mail.gmail.com>
 <75ACDDE0-48EF-466C-8DF8-6D7861A71815@instaclustr.com>
 <CA+VSrLq6oX8eR=ei_4B=WK+9iZF88QYho-GxJYpBL2x9Om1y0g@mail.gmail.com>
 <CAFqWSgdi4gC1e7NTwjJRcaHvzn39MuexfeKVfBA9oYuDDJ25wA@mail.gmail.com>
 <CAKmMYa8KvMFt-Kp2=boFBcWCWsAChJJt4F3UHkugL4xJNLanWw@mail.gmail.com>
Subject: Re: EBS SSD <-> Cassandra ?
X-Mailer: Airmail (237)
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="53a34a88_19495cff_3cd"
X-Virus-Checked: Checked by ClamAV on apache.org

--53a34a88_19495cff_3cd
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

does an elastic network interface really use a different physical network=
 interface=3F or is it just to give the ability for multiple ip addresses=
=3F



On June 19, 2014 at 3:56:34 PM, Nate McCall (nate=40thelastpickle.com) wr=
ote:

If someone really wanted to try this it, I recommend adding an Elastic Ne=
twork Interface or two for gossip and client/API traffic. This lets EBS a=
nd management traffic have the pre-configured network.=C2=A0


On Thu, Jun 19, 2014 at 6:54 AM, Benedict Elliott Smith <belliottsmith=40=
datastax.com> wrote:
I would say this is worth benchmarking before jumping to conclusions. The=
 network being a bottleneck (or latency causing) for EBS is, to my knowle=
dge, supposition, and instances can be started with direct connections to=
 EBS if this is a concern. The blog post below shows that even without SS=
Ds the EBS-optimised provisioned-IOPS instances show pretty consistent la=
tency numbers, although those latencies are higher than you would typical=
ly expect from locally attached storage.

http://blog.parse.com/2012/09/17/parse-databases-upgraded-to-amazon-provi=
sioned-iops/

Note, I'm not endorsing the use of EBS. Cassandra is designed to scale up=
 with number of nodes, not with depth of nodes (as Ben mentions, saturati=
ng a single node's data capacity is pretty easy these days. CPUs rapidly =
become the bottleneck as you try to go deep). However the argument that E=
BS cannot provide consistent performance seems overly pessimistic, and sh=
ould probably be empirically determined for your use case.


On Thu, Jun 19, 2014 at 9:50 AM, Alain RODRIGUEZ <arodrime=40gmail.com> w=
rote:
Ok, looks fair enough.

Thanks guys. I would be great to be able to add disks when amount of data=
 raises and add nodes when throughput increases... :)


2014-06-19 5:27 GMT+02:00 Ben Bromhead <ben=40instaclustr.com>:

http://www.datastax.com/documentation/cassandra/1.2/cassandra/architectur=
e/architecturePlanningEC2=5Fc.html

=46rom the link:

EBS volumes are not recommended for Cassandra data volumes for the follow=
ing reasons:

=E2=80=A2 EBS volumes contend directly for network throughput with standa=
rd packets. This means that EBS throughput is likely to fail if you satur=
ate a network link.
=E2=80=A2 EBS volumes have unreliable performance. I/O performance can be=
 exceptionally slow, causing the system to back load reads and writes unt=
il the entire cluster becomes unresponsive.
=E2=80=A2 Adding capacity by increasing the number of EBS volumes per hos=
t does not scale. You can easily surpass the ability of the system to kee=
p effective buffer caches and concurrently serve=C2=A0requests for all of=
 the data it is=C2=A0responsible for managing.

Still applies, especially the network contention and latency issues.=C2=A0=


Ben Bromhead
Instaclustr =7C=C2=A0www.instaclustr.com=C2=A0=7C=C2=A0=40instaclustr=C2=A0=
=7C +61 415 936 359

On 18 Jun 2014, at 7:18 pm, Daniel Chia <danchia=40coursera.org> wrote:

While they guarantee IOPS, they don't really make any guarantees about la=
tency. Since EBS goes over the network, there's so many things in the pat=
h of getting at your data, I would be concerned with random latency spike=
s, unless proven otherwise.

Thanks,
Daniel


On Wed, Jun 18, 2014 at 1:58 AM, Alain RODRIGUEZ <arodrime=40gmail.com> w=
rote:
In this document it is said :

Provisioned IOPS (SSD)=C2=A0- Volumes of this type are ideal for the most=
 demanding I/O intensive, transactional workloads and large relational or=
 NoSQL databases. This volume type provides the most consistent performan=
ce and allows you to provision the exact level of performance you need wi=
th the most predictable and consistent performance. With this type of vol=
ume you provision exactly what you need, and pay for what you provision. =
Once again, you can achieve up to 48,000 IOPS by connecting multiple volu=
mes together using RAID.


2014-06-18 10:57 GMT+02:00 Alain RODRIGUEZ <arodrime=40gmail.com>:

Hi,

I just saw this :=C2=A0http://aws.amazon.com/fr/blogs/aws/new-ssd-backed-=
elastic-block-storage/

Since the problem with EBS was the network, there is no chance that this =
hardware architecture might be useful alongside Cassandra, right =3F

Alain








--
-----------------
Nate McCall
Austin, TX
=40zznate

Co-=46ounder & Sr. Technical Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
--53a34a88_19495cff_3cd
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

<html><head><style>body=7Bfont-family:Helvetica,Arial;font-size:13px=7D</=
style></head><body style=3D=22word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space;=22><div id=3D=22bloop=5Fcust=
omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r=
gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>does an elastic networ=
k interface really use a different physical network interface=3F or is it=
 just to give the ability for multiple ip addresses=3F</div> <div id=3D=22=
bloop=5Fsign=5F1403210336544628224=22 class=3D=22bloop=5Fsign=22><br><br>=
<span style=3D=22font-family:helvetica,arial;font-size:13px=22></span><sp=
an></span></div> <br><p style=3D=22color:=23000;=22>On June 19, 2014 at 3=
:56:34 PM, Nate McCall (<a href=3D=22mailto:nate=40thelastpickle.com=22>n=
ate=40thelastpickle.com</a>) wrote:</p> <blockquote type=3D=22cite=22 cla=
ss=3D=22clean=5Fbq=22><span><div><div></div><div>


<title></title>


<div dir=3D=22ltr=22>If someone really wanted to try this it, I recommend=

adding an Elastic Network Interface or two for gossip and
client/API traffic. This lets EBS and management traffic have the
pre-configured network.&nbsp;</div>
<div class=3D=22gmail=5Fextra=22><br>
<br>
<div class=3D=22gmail=5Fquote=22>On Thu, Jun 19, 2014 at 6:54 AM, Benedic=
t
Elliott Smith <span dir=3D=22ltr=22>&lt;<a href=3D=22mailto:belliottsmith=
=40datastax.com=22 target=3D=22=5Fblank=22>belliottsmith=40datastax.com</=
a>&gt;</span> wrote:<br>
<blockquote class=3D=22gmail=5Fquote=22 style=3D=22margin:0 0 0 .8ex;bord=
er-left:1px =23ccc solid;padding-left:1ex=22>
<div dir=3D=22ltr=22>I would say this is worth benchmarking before
jumping to conclusions. The network being a bottleneck (or latency
causing) for EBS is, to my knowledge, supposition, and instances
can be started with direct connections to EBS if this is a concern.
The blog post below shows that even without SSDs the EBS-optimised
provisioned-IOPS instances show pretty consistent latency numbers,
although those latencies are higher than you would typically expect
from locally attached storage.
<div><br></div>
<div><a href=3D=22http://blog.parse.com/2012/09/17/parse-databases-upgrad=
ed-to-amazon-provisioned-iops/=22 target=3D=22=5Fblank=22>http://blog.par=
se.com/2012/09/17/parse-databases-upgraded-to-amazon-provisioned-iops/</a=
><br>
</div>
<div><br></div>
<div>Note, I'm not endorsing the use of EBS. Cassandra is designed
to scale up with number of nodes, not with depth of nodes (as Ben
mentions, saturating a single node's data capacity is pretty easy
these days. CPUs rapidly become the bottleneck as you try to go
deep). However the argument that EBS cannot provide consistent
performance seems overly pessimistic, and should probably be
empirically determined for your use case.</div>
</div>
<div class=3D=22HOEnZb=22>
<div class=3D=22h5=22>
<div class=3D=22gmail=5Fextra=22><br>
<br>
<div class=3D=22gmail=5Fquote=22>On Thu, Jun 19, 2014 at 9:50 AM, Alain
RODRIGUEZ <span dir=3D=22ltr=22>&lt;<a href=3D=22mailto:arodrime=40gmail.=
com=22 target=3D=22=5Fblank=22>arodrime=40gmail.com</a>&gt;</span> wrote:=
<br>
<blockquote class=3D=22gmail=5Fquote=22 style=3D=22margin:0 0 0 .8ex;bord=
er-left:1px =23ccc solid;padding-left:1ex=22>
<div dir=3D=22ltr=22>Ok, looks fair enough.
<div><br></div>
<div>Thanks guys. I would be great to be able to add disks when
amount of data raises and add nodes when throughput increases...
:)</div>
</div>
<div class=3D=22gmail=5Fextra=22><br>
<br>
<div class=3D=22gmail=5Fquote=22>2014-06-19 5:27 GMT+02:00 Ben Bromhead
<span dir=3D=22ltr=22>&lt;<a href=3D=22mailto:ben=40instaclustr.com=22 ta=
rget=3D=22=5Fblank=22>ben=40instaclustr.com</a>&gt;</span>:
<div>
<div><br>
<blockquote class=3D=22gmail=5Fquote=22 style=3D=22margin:0 0 0 .8ex;bord=
er-left:1px =23ccc solid;padding-left:1ex=22>
<div style=3D=22word-wrap:break-word=22><a href=3D=22http://www.datastax.=
com/documentation/cassandra/1.2/cassandra/architecture/architecturePlanni=
ngEC2=5Fc.html=22 target=3D=22=5Fblank=22>http://www.datastax.com/documen=
tation/cassandra/1.2/cassandra/architecture/architecturePlanningEC2=5Fc.h=
tml</a>
<div><br></div>
<div>=46rom the link:</div>
<div><br></div>
<div>EBS volumes are not recommended for Cassandra data volumes for
the following reasons:<br>
<br>
<div>=E2=80=A2 EBS volumes contend directly for network throughput with
standard packets. This means that EBS throughput is likely to fail
if you saturate a network link.<br></div>
<div>=E2=80=A2 EBS volumes have unreliable performance. I/O performance c=
an
be exceptionally slow, causing the system to back load reads and
writes until the entire cluster becomes unresponsive.<br></div>
<div>=E2=80=A2 Adding capacity by increasing the number of EBS volumes pe=
r
host does not scale. You can easily surpass the ability of the
system to keep effective buffer caches and concurrently
serve&nbsp;requests for all of the data it is&nbsp;responsible for
managing.</div>
</div>
<div><br></div>
<div>Still applies, especially the network contention and latency
issues.&nbsp;</div>
<div><span><font color=3D=22=23888888=22><br></font></span>
<div>
<div>
<div>
<div><span><font color=3D=22=23888888=22>Ben Bromhead</font></span></div>=

</div>
<div><span><font color=3D=22=23888888=22>Instaclustr =7C&nbsp;<a href=3D=22=
https://www.instaclustr.com/=22 target=3D=22=5Fblank=22>www.instaclustr.c=
om</a>&nbsp;=7C&nbsp;<a href=3D=22http://twitter.com/instaclustr=22 targe=
t=3D=22=5Fblank=22>=40instaclustr</a>&nbsp;=7C <a href=3D=22tel:%2B61%204=
15%20936%20359=22 value=3D=22+61415936359=22 target=3D=22=5Fblank=22>+61 =
415 936 359</a></font></span></div>
</div>
</div>
<div>
<div><br>
<div>
<div>On 18 Jun 2014, at 7:18 pm, Daniel Chia &lt;<a href=3D=22mailto:danc=
hia=40coursera.org=22 target=3D=22=5Fblank=22>danchia=40coursera.org</a>&=
gt; wrote:</div>
<br>
<blockquote type=3D=22cite=22>
<div dir=3D=22ltr=22>While they guarantee IOPS, they don't really make
any guarantees about latency. Since EBS goes over the network,
there's so many things in the path of getting at your data, I would
be concerned with random latency spikes, unless proven
otherwise.</div>
<div class=3D=22gmail=5Fextra=22><br clear=3D=22all=22>
<div>Thanks,
<div>Daniel</div>
</div>
<br>
<br>
<div class=3D=22gmail=5Fquote=22>On Wed, Jun 18, 2014 at 1:58 AM, Alain
RODRIGUEZ <span dir=3D=22ltr=22>&lt;<a href=3D=22mailto:arodrime=40gmail.=
com=22 target=3D=22=5Fblank=22>arodrime=40gmail.com</a>&gt;</span> wrote:=
<br>
<blockquote class=3D=22gmail=5Fquote=22 style=3D=22margin:0 0 0 .8ex;bord=
er-left:1px =23ccc solid;padding-left:1ex=22>
<div dir=3D=22ltr=22>In this document it is said :
<div><br></div>
<div>
<ul style=3D=22padding:0px;margin:0px 0px 15px 30px;color:rgb(51,51,51);f=
ont-family:HelveticaNeue,Helvetica,Helvetica,Arial,sans-serif;font-size:1=
4px;line-height:22.399999618530273px=22>
<li style=3D=22padding-bottom:6px=22><span style=3D=22font-family:Helveti=
caNeueBold,Helvetica,Helvetica,Arial,sans-serif=22>
Provisioned IOPS (SSD)</span>&nbsp;- Volumes of this type are ideal
for the most demanding I/O intensive, transactional workloads and
large relational or NoSQL databases. This volume type provides the
most consistent performance and allows you to provision the exact
level of performance you need with the most predictable and
consistent performance. With this type of volume you provision
exactly what you need, and pay for what you provision. Once again,
you can achieve up to 48,000 IOPS by connecting multiple volumes
together using RAID.</li>
</ul>
</div>
</div>
<div class=3D=22gmail=5Fextra=22><br>
<br>
<div class=3D=22gmail=5Fquote=22>2014-06-18 10:57 GMT+02:00 Alain RODRIGU=
EZ
<span dir=3D=22ltr=22>&lt;<a href=3D=22mailto:arodrime=40gmail.com=22 tar=
get=3D=22=5Fblank=22>arodrime=40gmail.com</a>&gt;</span>:
<div>
<div><br>
<blockquote class=3D=22gmail=5Fquote=22 style=3D=22margin:0 0 0 .8ex;bord=
er-left:1px =23ccc solid;padding-left:1ex=22>
<div dir=3D=22ltr=22>Hi,
<div><br></div>
<div>I just saw this :&nbsp;<a href=3D=22http://aws.amazon.com/fr/blogs/a=
ws/new-ssd-backed-elastic-block-storage/=22 target=3D=22=5Fblank=22>http:=
//aws.amazon.com/fr/blogs/aws/new-ssd-backed-elastic-block-storage/</a></=
div>
<div><br></div>
<div>Since the problem with EBS was the network, there is no chance
that this hardware architecture might be useful alongside
Cassandra, right =3F</div>
<div><span><font color=3D=22=23888888=22><br></font></span></div>
<div><span><font color=3D=22=23888888=22>Alain</font></span></div>
</div>
</blockquote>
</div>
</div>
</div>
<br></div>
</blockquote>
</div>
<br></div>
</blockquote>
</div>
<br></div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
<br></div>
</blockquote>
</div>
<br></div>
</div>
</div>
</blockquote>
</div>
<br>
<br clear=3D=22all=22>
<div><br></div>
--<br>
<div dir=3D=22ltr=22>-----------------<br>
Nate McCall<br>
Austin, TX<br>
=40zznate<br>
<br>
Co-=46ounder &amp; Sr. Technical Consultant<br>
Apache Cassandra Consulting<br>
<a href=3D=22http://www.thelastpickle.com=22 target=3D=22=5Fblank=22>http=
://www.thelastpickle.com</a></div>
</div>


</div></div></span></blockquote></body></html>
--53a34a88_19495cff_3cd--