Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of scottblanc@gmail.com
 designates 209.85.212.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=F4ddSw8i+uNxjXwmV21PFkw1BpFTT0AGyd2vWp2TVKwXne75yInfZT0oX/+3Rtq+GV
         Ld66Wz2I1C54v9f/Iq18bVTSJ1nBeSfRQAs/I1gL08+2C14ihFGavLp2lwCHa+3Dvyhj
         1mEjeC2WHgS9U4MH5IYpBDNEaD/+GHlEvcH+Y=
MIME-Version: 1.0
Date: Tue, 13 Apr 2010 09:23:39 -0700
Message-ID: <h2x401b577e1004130923z7c20f80cuee0beabf18da9ce0@mail.gmail.com>
Subject: Internal error
From: Scott White <scottblanc@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=00c09f8a4d9e325675048420af7c

--00c09f8a4d9e325675048420af7c
Content-Type: text/plain; charset=ISO-8859-1

We started seeing for the first time errors that look like this last night
after things were running smoothly for a couple days after switching to
0.6-rc1:

ERROR [pool-1-thread-60] 2010-04-13 04:58:11,806 Cassandra.java (line 1492)
Internal error processing insert

We had not made any other changes to cassandra server or client config in
days. Operationally we did try to bootstrap two new nodes, one of which
failed after timing out. The other thing we did was a cleanup operation.

Is it possible somehow any of those operations/events would somehow get
things into a bad state? If not, any other ideas of what to look for as the
root cause?

Overall, the cluster seemed relatively healthy with the exception that one
of our nodes which happens to be the seed node has been having excessively
high network delays, hovering around 30-40 ms although it's been in that
state for more than a week.

Also a side question I was curious about, is the state of a boostrap or
cleanup operation maintained after restart?

thanks,
Scott

--00c09f8a4d9e325675048420af7c
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

We started seeing for the first time errors that look like this last night =
after things were running smoothly for a couple days after switching to 0.6=
-rc1:<br><br>ERROR [pool-1-thread-60] 2010-04-13 04:58:11,806 Cassandra.jav=
a (line=20
1492) Internal error processing insert<br><br>We had not made any other cha=
nges to cassandra server or client config in days. Operationally we did try=
 to bootstrap two new nodes, one of which failed after timing out. The othe=
r thing we did was a cleanup operation.<br>
<br>Is it possible somehow any of those operations/events would somehow get=
 things into a bad state? If not, any other ideas of what to look for as th=
e root cause?<br><br>Overall, the cluster seemed relatively healthy with th=
e exception that one of our nodes which happens to be the seed node has bee=
n having excessively high network delays, hovering around 30-40 ms although=
 it&#39;s been in that state for more than a week. <br>
<br>Also a side question I was curious about, is the state of a boostrap or=
 cleanup operation maintained after restart? <br><br>thanks,<br>Scott<br>

--00c09f8a4d9e325675048420af7c--