Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1F857D3B1 for ; Fri, 7 Dec 2012 21:04:44 +0000 (UTC) Received: (qmail 3595 invoked by uid 500); 7 Dec 2012 21:04:41 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 3568 invoked by uid 500); 7 Dec 2012 21:04:41 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 3559 invoked by uid 99); 7 Dec 2012 21:04:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Dec 2012 21:04:41 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ailinykh@gmail.com designates 209.85.215.44 as permitted sender) Received: from [209.85.215.44] (HELO mail-la0-f44.google.com) (209.85.215.44) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Dec 2012 21:04:36 +0000 Received: by mail-la0-f44.google.com with SMTP id d3so735822lah.31 for ; Fri, 07 Dec 2012 13:04:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=uGh47+c7WTl6wXsDKdNWbYzv1rV+L04BhrKGNMEUuKg=; b=UnWCmXFR5CntN0O/eElW7Z7vzjYLc9jS1op9nnbfMOav4ZQ9r14FhgnOm+7KG/nTzC HQzLuvre7Pa1hs2np9oNbnfFPubYqR5MxjdyPYPkeE9t6kHtfHLX0li9KnDCRBpklu0f Ps0Leb9b1Qf1CUO6qHDtzJhP+lrkqmUInzMnLhx6n5Vip+wPNpnhh1ZKJpbbND7/rsgz u97s+laveXYYw7TrWzWHptfURlfrnv+GPSsIU7cGEXfOH9ylxmiCo1C8TrA8z1mU6s3X OX7rNjcMCxHqIgJMY4afv1pXFpaekAqvHKjc367ozFjNNaTcbtm2S0qGra4Xf77iJEQK CHOg== MIME-Version: 1.0 Received: by 10.152.46.161 with SMTP id w1mr6643963lam.27.1354914255071; Fri, 07 Dec 2012 13:04:15 -0800 (PST) Received: by 10.114.28.34 with HTTP; Fri, 7 Dec 2012 13:04:15 -0800 (PST) In-Reply-To: References: Date: Fri, 7 Dec 2012 13:04:15 -0800 Message-ID: Subject: Re: how to take consistant snapshot? From: Andrey Ilinykh To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=bcaec5540890dfeb3c04d0498ed1 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec5540890dfeb3c04d0498ed1 Content-Type: text/plain; charset=ISO-8859-1 Agreed. On Fri, Dec 7, 2012 at 12:38 PM, Tyler Hobbs wrote: > Right. I don't personally think incremental backup is useful beyond > restoring individual nodes unless none of your data happens to reference > any other rows. > > > On Fri, Dec 7, 2012 at 11:37 AM, Andrey Ilinykh wrote: > >> That's right. But when I have incremental backup on each CF gets flushed >> independently. I have "hot" CF which gets flushed every several minutes and >> regular CF which gets flushed every hour or so. They have references to >> each other and data in sstables is definitely inconsistent. >> >> >> >> On Fri, Dec 7, 2012 at 9:28 AM, Tyler Hobbs wrote: >> >>> Snapshots trigger a flush first, so data that's currently in the commit >>> log will be covered by the snapshot. >>> >>> >>> On Thu, Dec 6, 2012 at 11:52 PM, Andrey Ilinykh wrote: >>> >>>> >>>> >>>> >>>> On Thu, Dec 6, 2012 at 7:34 PM, aaron morton wrote: >>>> >>>>> For background >>>>> >>>>> >>>>> http://wiki.apache.org/cassandra/Operations?highlight=%28snapshot%29#Consistent_backups >>>>> >>>>> If you it for a single node then yes there is a chance of >>>>> inconsistency across CF's. >>>>> >>>>> If you have mulitple nodes the snashots you take on the later nodes >>>>> will help. If you use CL QUOURM for reads you *may* be ok (cannot work it >>>>> out quickly.). If you use CL ALL for reads you will be ok. Or you can use >>>>> nodetool repair to ensure the data is consistent. >>>>> >>>>> I'm talking about restoring whole cluster, so all nodes are restored >>>> from backup and all of them are inconsistent because they lost data from >>>> commit logs. It doesn't matter what CL I use, some data may be lost. >>>> Cassandra 1.1 supports commit log archiving >>>> http://www.datastax.com/docs/1.1/configuration/commitlog_archiving >>>> I think if I store both flushed sstables and commit logs it should >>>> solve my problem. I'm wondering if someone has any experience with this >>>> feature? >>>> >>>> Thank you, >>>> Andrey >>>> >>> >>> >>> >>> -- >>> Tyler Hobbs >>> DataStax >>> >>> >> > > > -- > Tyler Hobbs > DataStax > > --bcaec5540890dfeb3c04d0498ed1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Agreed.



On Fri, Dec 7, 2012 at 12:38 PM, Tyler Hobbs <= ;tyler@datastax.com= > wrote:
Right.=A0 I don't personally think incre= mental backup is useful beyond restoring individual nodes unless none of yo= ur data happens to reference any other rows.


On Fri, Dec 7, 2012 at 11:37 AM, Andrey Ilinykh <ailinykh@gmail.com&g= t; wrote:
That's right. But when I have incremental backup on each CF gets flushe= d independently. I have "hot" CF which gets flushed every several= minutes and regular CF which gets flushed every hour or so. They have refe= rences to each other and data in sstables is=A0definitely=A0inconsistent.



On = Fri, Dec 7, 2012 at 9:28 AM, Tyler Hobbs <tyler@datastax.com> wrote:
Snapshots trigger a flush first, so data tha= t's currently in the commit log will be covered by the snapshot.


On Thu, Dec 6, 2012 at 11:52 P= M, Andrey Ilinykh <ailinykh@gmail.com> wrote:



On Thu, Dec 6, 2012 at 7:34 PM, aaron morton <aaron@thelastpickle.com> wrote:
For background


If you it for a single node then yes there is a chance = of inconsistency across CF's.=A0

If you have m= ulitple nodes the snashots you take on the later nodes will help. If you us= e CL QUOURM for reads you *may* be ok (cannot work it out quickly.). If you= use CL ALL for reads you will be ok. Or you can use nodetool repair to ens= ure the data is consistent.=A0

I'm talking about restorin= g whole cluster, so all nodes are restored from backup and all of them are= =A0inconsistent because they lost data =A0from commit logs. =A0It doesn'= ;t matter what CL I use, some data may be lost.=A0
Cassandra 1.1 supports commit log archiving

<= /div>



--
Tyler Hobbs
DataStax
<= br>




--
Tyl= er Hobbs
DataStax
<= br>
--bcaec5540890dfeb3c04d0498ed1--