Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DDC1718121 for ; Fri, 13 Nov 2015 17:57:37 +0000 (UTC) Received: (qmail 72548 invoked by uid 500); 13 Nov 2015 17:57:35 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 72503 invoked by uid 500); 13 Nov 2015 17:57:35 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 72493 invoked by uid 99); 13 Nov 2015 17:57:35 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Nov 2015 17:57:35 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id E050E180EA9 for ; Fri, 13 Nov 2015 17:57:34 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.122 X-Spam-Level: *** X-Spam-Status: No, score=3.122 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, KAM_HUGEIMGSRC=0.2, T_KAM_HTML_FONT_INVALID=0.01, T_REMOTE_IMAGE=0.01, URIBL_BLOCKED=0.001, URI_TRY_3LD=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=datastax.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id COmGEtDJzk6j for ; Fri, 13 Nov 2015 17:57:20 +0000 (UTC) Received: from mail-wm0-f52.google.com (mail-wm0-f52.google.com [74.125.82.52]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id D17D34450F for ; Fri, 13 Nov 2015 17:57:19 +0000 (UTC) Received: by wmec201 with SMTP id c201so93235228wme.0 for ; Fri, 13 Nov 2015 09:57:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=datastax.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=kB9a7MADpXP/rm2jGyrZzw1q46Jm+/KQrgEKuxCIssQ=; b=RPB+qTtq8ZZga/kHGca9KyDHJaHTcV/dWL1vUO/wQXUz2xcQ1fxV3ZhvQyGJy0k5Uz q7/3ze9IYIS1U/HYvJ3MQFE7jzVkvF8v6SNToj+UhH/zWogR401m/xejbyzwSF8IqFay jBVOI7SQ+ulHGCJE4HCGScTsW1kKG5StGLvuU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=kB9a7MADpXP/rm2jGyrZzw1q46Jm+/KQrgEKuxCIssQ=; b=ePKcEBV4QyXX2K86ppIY/dB9pDgijMwCHkuYu/y66ibzZyVgKy7bjnl2mnHSFYXYCZ fABPoLoaQl6SnQ2a3PiWwZy4eyYm9UGYF8QPuo1iX/38TqwVqE1WWGcT4ZweuQTBJb+r SSlp6VKTidq5CMEy6tgKEpry7KlHs3KvD8qjHIXu66bhlOsGq+mKkTOv8UR0RoQj1zut Ly2/oQT2cJQ+GO3X7ggSg352fO4VPolUxfrrBwy7tL270Ex66gcoy+pyJIwjj86VzcgQ bA9EEvANzm6Lvoi9ezN5rojv3zR+YItN/abQoLRHpcBVZkFPZO09nyGU/BKGTZVYGxkw fn/A== X-Gm-Message-State: ALoCoQnrkaFpASd/hVJ5EDkv0GPlqAGVL/xzJH8MZOsf9i78+/MzwbWsGFWmnxLvWK/q3+W/iqo5 X-Received: by 10.28.226.86 with SMTP id z83mr5525699wmg.77.1447437438792; Fri, 13 Nov 2015 09:57:18 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.157.10 with HTTP; Fri, 13 Nov 2015 09:56:59 -0800 (PST) In-Reply-To: References: From: Sebastian Estevez Date: Fri, 13 Nov 2015 12:56:59 -0500 Message-ID: Subject: Re: UnknownColumnFamily exception / schema inconsistencies To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001a114b0f6e5fd5d305246fca33 --001a114b0f6e5fd5d305246fca33 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I think you're just missing the steps in *Bold*: If THERE ARE TWO OR MORE DIRECTORIES: 4)Identify from schema_column_families which cf ID is the "new" one (currently in use). cqlsh -e "select * from system.schema_column_families"|grep *5) Move the data from the "old" one to the "new" one and remove the old directory. * *6) If there are multiple "old" ones repeat 5 for every "old" directory.* *7) run nodetool refresh* All the best, [image: datastax_logo.png] Sebasti=C3=A1n Est=C3=A9vez Solutions Architect | 954 905 8615 | sebastian.estevez@datastax.com [image: linkedin.png] [image: facebook.png] [image: twitter.png] [image: g+.png] DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world=E2=80=99s most innovative enterpri= ses. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay. On Fri, Nov 13, 2015 at 12:37 PM, Maciek Sakrejda wrote= : > Any advice on how to proceed here? Sebastian seems to have guessed > correctly at the underlying issue, but I'm still not sure how to resolve > this given what I see in the data directory and the catalogs. > > On Wed, Nov 11, 2015 at 12:15 PM, Maciek Sakrejda > wrote: > >> On Wed, Nov 11, 2015 at 9:55 AM, Sebastian Estevez < >> sebastian.estevez@datastax.com> wrote: >> >>> Stupid question, but how do I find the problem table? The error message >>>> complains about a keyspace (by uuid); I haven't seen errors relating t= o a >>>> specific table. I've poked around in the data directory, but I'm not s= ure >>>> what I'm looking for. >>> >>> >>> Is the message complaining about a *keyspace* or abou*t a table (cfid)*= ? >>> You'r original was complaining about a table: >>> >> >>> at=3DIncomingTcpConnection.run UnknownColumnFamilyException reading fro= m >>>> socket; closing org.apache.cassandra.db.UnknownColumnFamilyException: >>>> Couldn't find *cfId=3D3ecce750-84d3-11e5-bdd9-**dd7717dcdbd5* >>> >>> >> Sorry, you're absolutely right--it's the table from this error message. = I >> confused myself. But now I was able to find it: >> >> cursors-3ecce75084d311e5bdd9dd7717dcdbd5 >> cursors-3ed23e8084d311e583b30fc0205655f5 >> >> The second uuid is the one that shows up via the schema_columnfamilies >> query, but on two of the nodes, the directory with the *other* uuid exis= ts. >> Can I just rename the directory on these two nodes? Or how should I proc= eed? >> > > --001a114b0f6e5fd5d305246fca33 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable 
I thi=
nk you're just missing the steps in Bold:

If THERE ARE TWO OR MORE DIRECTORIES:
4)Identify from sc=
hema_column_families which=C2=A0cf=C2=A0ID=C2=A0is the "new" one =
(currently in use).=C2=A0
cqlsh -=
e "select * from system.schema_column_families"|grep <table na=
me>
5) Move the data from t=
he "old" one to the "new" one and remove the old direct=
ory.=C2=A0
6) If there are multiple "old" ones repeat 5 for every "old=
" directory.
7) run nodetool refresh

=


All the best,

3D"datastax_logo.png"<=
/span>
Solutions Architect | 954 90=
5 8615 | sebastian.estevez@datastax.com
3D"=  3D"twitter.png" 3D"g+.png" 

3D""=


Data=
Stax is the fastest, most scalable distributed database technology, delivering A=
pache Cassandra to the world=E2=80=99s most innovative enterprises. Datasta=
x is built to be agile, always-on, and predictably scalable to any size. Wi=
th more than 500 customers in 45 countries, DataS=
tax is the database technology and transactional backbone of choice for the=
 worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.=
 
=



On Fri, Nov 13, 2015 at 12:37 PM, Maciek Sak=
rejda <maciek@heroku.com> wrote:
Any advice on how to proceed here? Sebastian seems =
to have guessed correctly at the underlying issue, but I'm still not su=
re how to resolve this given what I see in the data directory and the catal=
ogs.

On Wed, Nov 11, 2015 at 12:15 PM, Maci=
ek Sakrejda <maciek@heroku.com> wrote:
On Wed, Nov 11, 2015 at 9:55 AM, Sebas=
tian Estevez <sebastian.estevez@datastax.com> w=
rote:
<=
blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l=
eft:1px solid rgb(204,204,204);padding-left:1ex">Stupid question, but how do I find the problem table? The error mess=
age complains about a keyspace (by uuid); I haven't seen errors relatin=
g to a specific table. I've poked around in the data directory, but I&#=
39;m not sure what I'm looking for.

<=
/span>Is the message complaining about a keyspace or about a tabl=
e (cfid)? You'r original was complaining about a table:
<=
span>

at=3DIncomingTcpConnection.run UnknownColumnFam=
ilyException reading from socket; closing org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't f=
ind cfId=3D3ecce750-84d3-11e5-bdd9-dd7717dcdbd5

Sorry, you're absolutely right--it's the ta=
ble from this error message. I confused myself. But now I was able to find =
it:

cursors-3ecce75084d311e5bdd9dd7717dcdbd5
cursors-3ed23e8084d3=
11e583b30fc0205655f5

The second uuid is the one that shows up via th=
e schema_columnfamilies query, but on two of the nodes, the directory with =
the *other* uuid exists. Can I just rename the directory on these two nodes=
? Or how should I proceed?







--001a114b0f6e5fd5d305246fca33--