Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A6FC6999A for ; Wed, 21 Sep 2011 09:21:03 +0000 (UTC) Received: (qmail 3347 invoked by uid 500); 21 Sep 2011 09:21:01 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 3313 invoked by uid 500); 21 Sep 2011 09:21:01 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 3305 invoked by uid 99); 21 Sep 2011 09:21:01 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Sep 2011 09:21:01 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.160.172] (HELO mail-gy0-f172.google.com) (209.85.160.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Sep 2011 09:20:53 +0000 Received: by gyd12 with SMTP id 12so1205037gyd.31 for ; Wed, 21 Sep 2011 02:20:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.151.157.10 with SMTP id j10mr745709ybo.68.1316596832647; Wed, 21 Sep 2011 02:20:32 -0700 (PDT) Received: by 10.151.149.18 with HTTP; Wed, 21 Sep 2011 02:20:32 -0700 (PDT) Date: Wed, 21 Sep 2011 12:20:32 +0300 Message-ID: Subject: Read failure when adding node + move; Or: What is the right way to add a node? From: David Boxenhorn To: user Content-Type: multipart/alternative; boundary=000e0cd639c885bbd704ad7016f9 X-Virus-Checked: Checked by ClamAV on apache.org --000e0cd639c885bbd704ad7016f9 Content-Type: text/plain; charset=ISO-8859-1 Initial state: 3 nodes, RF=3, version = 0.7.8, some queries are with CL=QUORUM 1. Add node with with correct token for 4 nodes, repair 2. Move first node to balance 4 nodes, repair 3. Move second node ===> Start getting timeouts, Hector warning: WARNING - Error: me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be enough replicas present to handle consistency level. What is going on? My traffic isn't high. None of my nodes' logs show ANYTHING during the move 4. When the node finishes moving, the timeouts stop happening Is there some state in the above scenario that I don't have the required replication of at least 2? --000e0cd639c885bbd704ad7016f9 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Initial state: 3 nodes, RF=3D3, version =3D 0.7.8, some qu= eries are with CL=3DQUORUM

1. Add node with with correct token for 4= nodes, repair
2. Move first node to balance 4 nodes, repair
3. Move = second node

=3D=3D=3D> Start getting timeouts, Hector warning: WARNING - Error: = me.prettyprint.hector.api.exceptions.HUnavailableException: : May not be en= ough replicas present to handle consistency level.

What is going on?= My traffic isn't high. None of my nodes' logs show ANYTHING during= the move

4. When the node finishes moving, the timeouts stop happening

Is= there some state in the above scenario that I don't have the required = replication of at least 2?
--000e0cd639c885bbd704ad7016f9--