Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A81CA90A3 for ; Wed, 4 Jul 2012 01:56:53 +0000 (UTC) Received: (qmail 81391 invoked by uid 500); 4 Jul 2012 01:56:50 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 81357 invoked by uid 500); 4 Jul 2012 01:56:50 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 81346 invoked by uid 99); 4 Jul 2012 01:56:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jul 2012 01:56:50 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of dan.foody@gmail.com designates 209.85.216.172 as permitted sender) Received: from [209.85.216.172] (HELO mail-qc0-f172.google.com) (209.85.216.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jul 2012 01:56:42 +0000 Received: by qcac10 with SMTP id c10so3393535qca.31 for ; Tue, 03 Jul 2012 18:56:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:mime-version:content-type:subject:date:in-reply-to:to :references:message-id:x-mailer; bh=d7b+SP8/eGUMkqhKCI3GYIvb+uqtO3f0ic/HDY5K/7s=; b=PacOd3d3yS8VHfn5clYdRvSM809dV1EMOwiBS4Ad/weLF26s1GujWDqEnPjXGvF2pQ QAnzWUd0/r/7L7Z+mbFg2OZgrfjfRTdN+2jFdDqFe93MjSdnRBe6N39uN+MUkrLECHKV uRAPWVGL7HQ52VGsXVpTHrc2eRLZDhQ33yJ7N0X7gLN+Qmj+Wflr4Vm24H+jcXhR0/4c 4e2c5GtEaoO4xPJPtqzipTnn7ukDjx6zZZzG00oLGgaRRRkz6ddwHReEynvogYTrDsHu QuFmWolJnio7ybbitehPtLNKwJs6QpEGoF3OzMA9Ob4CKwYdpkMvVe+iHT28TToAI1Eh sqrQ== Received: by 10.229.135.9 with SMTP id l9mr9738165qct.38.1341366981955; Tue, 03 Jul 2012 18:56:21 -0700 (PDT) Received: from dan-foodys-air.home (pool-173-76-21-132.bstnma.fios.verizon.net. [173.76.21.132]) by mx.google.com with ESMTPS id he6sm40380422qab.13.2012.07.03.18.56.19 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 03 Jul 2012 18:56:20 -0700 (PDT) From: Dan Foody Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: multipart/alternative; boundary="Apple-Mail=_D2EFC395-C43F-492D-A3C5-60A59D465195" Subject: Re: Expanding Cassandra on EC2 with consistency Date: Tue, 3 Jul 2012 21:56:32 -0400 In-Reply-To: To: user@cassandra.apache.org References: Message-Id: <47D9EE5E-2AA3-4B79-88B6-A5D8C8390B0A@gmail.com> X-Mailer: Apple Mail (2.1278) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_D2EFC395-C43F-492D-A3C5-60A59D465195 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Hi Alex, Can you share what replication factor you're running? And, are you using ephemeral disks or EBS volumes? Thanks! - Dan On Jul 3, 2012, at 5:52 PM, Alex Major wrote: > Hi Mike, >=20 > We've run a small (4 node) cluster in the EU region since September = last year. We run across all 3 availability zones in the EU region, with = 2 nodes in one AZ and then a further node in each AZ. The latency = difference between running inside of and between AZ's has been minimal = in our experience.=20 >=20 > It's only when we've gone cross-region that there's been latency = problem. We temporarily ran a 9 node cluster across 3 regions, however = even then using local quoram the latency was better than the standard = datacenter - datacenter latency we're used to. >=20 > EC2Snitch is definitely the way to go in favour of NTS in my opinion. = NTS was a pain to get setup with the internal (private) IP address = setup, so much so that we never got it safely replicating the data as we = wanted. >=20 > Alex. >=20 > On Tue, Jul 3, 2012 at 2:16 PM, Michael Theroux = wrote: > Hello, >=20 > We are currently running a web application utilizing Cassandra on EC2. = Given the recent outages experienced with Amazon, we want to consider = expanding Cassandra across availability zones sooner rather than later. >=20 > We are trying to determine the optimal way to deploy Cassandra in this = deployment. We are researching the NetworkTopologyStrategy, and the = EC2Snitch. We are also interested in providing a high level of read or = write consistency, >=20 > My understanding is that the EC2Snitch recognizes availability zones = as racks, and regions as data-centers. This seems to be a common = configuration. However, if we were to want to utilize queries with a = READ or WRITE consistency of QUORUM, would there be a high possibility = that the communication necessary to establish a quorum, across = availability zones? >=20 > My understanding is that the NetworkTopologyStrategy attempts to = prefer replicas be stored on other racks within the datacenter, which = would equate to other availability zones in EC2. This implies to me = that in order to have the quorum of nodes necessary to achieve = consistency, that Cassandra will communicate with nodes across = availability zones. >=20 > First, is my understanding correct? Second, given the high latency = that can sometimes exists between availability zones, is this a problem, = and instead we should treat availability zones as data centers? >=20 > Ideally, we would be able to setup a situation where we could store = replicas across availability zones in case of failure, but establish a = high level of read or write consistency within a single availability = zone. >=20 > I appreciate your responses, > Thanks, > -Mike >=20 >=20 >=20 >=20 --Apple-Mail=_D2EFC395-C43F-492D-A3C5-60A59D465195 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 Hi = Alex,

Can you share what replication factor you're = running?
And, are you using ephemeral disks or EBS = volumes?

Thanks!

- = Dan



On Jul 3, 2012, at 5:52 PM, Alex Major wrote:

Hi = Mike,

We've run a small (4 node) cluster in the EU = region since September last year. We run across all = 3 availability zones in the EU region, with 2 nodes in one AZ = and then a further node in each AZ. The latency difference between = running inside of and between AZ's has been minimal in our = experience. 

It's only when we've gone cross-region that there's = been latency problem. We temporarily ran a 9 node cluster across 3 = regions, however even then using local quoram the latency was better = than the standard datacenter - datacenter latency we're used to.

EC2Snitch is definitely the way to go in = favour of NTS in my opinion. NTS was a pain to get setup with the = internal (private) IP address setup, so much so that we never got it = safely replicating the data as we wanted.

Alex.

On = Tue, Jul 3, 2012 at 2:16 PM, Michael Theroux <mtheroux2@yahoo.com> wrote:
Hello,

We are currently running a web application utilizing Cassandra on EC2. =  Given the recent outages experienced with Amazon, we want to = consider expanding Cassandra across availability zones sooner rather = than later.

We are trying to determine the optimal way to deploy Cassandra in this = deployment.  We are researching the NetworkTopologyStrategy, and = the EC2Snitch.  We are also interested in providing a high level of = read or write consistency,

My understanding is that the EC2Snitch recognizes availability zones as = racks, and regions as data-centers.  This seems to be a common = configuration.  However, if we were to want to utilize queries with = a READ or WRITE consistency of QUORUM, would there be a high possibility = that the communication necessary to establish a quorum, across = availability zones?

My understanding is that the NetworkTopologyStrategy attempts to prefer = replicas be stored on other racks within the datacenter, which would = equate to other availability zones in EC2.  This implies to me that = in order to have the quorum of nodes necessary to achieve consistency, = that Cassandra will communicate with nodes across availability = zones.

First, is my understanding correct?  Second, given the high latency = that can sometimes exists between availability zones, is this a problem, = and instead we should treat availability zones as data centers?

Ideally, we would be able to setup a situation where we could store = replicas across availability zones in case of failure, but establish a = high level of read or write consistency within a single availability = zone.

I appreciate your responses,
Thanks,
-Mike





= --Apple-Mail=_D2EFC395-C43F-492D-A3C5-60A59D465195--