Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 87B06DBC4 for ; Wed, 8 Aug 2012 13:04:22 +0000 (UTC) Received: (qmail 81671 invoked by uid 500); 8 Aug 2012 13:04:20 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 81646 invoked by uid 500); 8 Aug 2012 13:04:20 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 81638 invoked by uid 99); 8 Aug 2012 13:04:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Aug 2012 13:04:20 +0000 X-ASF-Spam-Status: No, hits=0.0 required=5.0 tests=FSL_RCVD_USER,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ajwytzes@wise-guys.nl designates 194.109.125.200 as permitted sender) Received: from [194.109.125.200] (HELO mx1.wise-guys.nl) (194.109.125.200) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 08 Aug 2012 13:04:15 +0000 Received: from vito.wise-guys.nl (vito.wise-guys.nl [194.109.125.200]) by mx1.wise-guys.nl (8.14.4/8.13.7) with ESMTP id q78D3rjQ002695 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 8 Aug 2012 15:03:53 +0200 Received: (from aj@localhost) by vito.wise-guys.nl (8.14.4/8.13.7/Submit) id q78D3rvN002679 for user@cassandra.apache.org; Wed, 8 Aug 2012 15:03:53 +0200 Date: Wed, 8 Aug 2012 15:03:52 +0200 From: Arend-Jan Wijtzes To: user@cassandra.apache.org Subject: lost keyspace Message-ID: <20120808130352.GC25746@vito.wise-guys.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Checked: Checked by ClamAV on apache.org Hi, Today we rebooted a node in our cluster for maintenance and after that one of the keyspaces went missing. This is what we did leading up to this: nodetool drain -h localhost reboot restart cassandra There was an error during startup: ERROR [MutationStage:109] 2012-08-08 14:05:56,443 RowMutationVerbHandler.java (line 61) Error in row mutation org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find cfId=1014 (then we found keyspace twitter is missing) The sstables are still there so I tried to re-create the keyspace: cassandra-cli -h localhost [default@unknown] create keyspace twitter with placement_strategy = 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options = {replication_factor:3} ; b890b65a-40a6-3fc5-838d-34d58368cb64 Waiting for schema agreement... ... schemas agree across the cluster NotFoundException() default@unknown] use twitter; Keyspace 'twitter' not found. So we can't create the keyspace. The cassandra version is 1.1.0. What is the proper way to deal with this? Thanks for your help. Arend-Jan -- Arend-Jan Wijtzes -- Wiseguys -- www.wise-guys.nl