Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of vram.kouramajian@gmail.com
 designates 209.85.220.172 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=Yz/4Hjo6YRM6tYmxlrl494584cFq7pLxYZEvzNB1a1UxYU5f9eUtHhHg841clcVtgm
         vaH4HQ2dVsaSDFlZEqNUGGolCL0QJgiIBN4r9w5M+DX60roDbnxzxhXz8Jn71pSTH0/M
         TQPpnYWCGFBE/Onv0JERe2hxHMCOXO/EojuMw=
MIME-Version: 1.0
In-Reply-To: <301C715B-CD4A-4C09-9E4A-A2F04635D937@joestump.net>
References: <BANLkTin_eCymOmsV_oL-X874txBK8cOTgg@mail.gmail.com>
	<301C715B-CD4A-4C09-9E4A-A2F04635D937@joestump.net>
Date: Sat, 9 Apr 2011 12:11:16 -0700
Message-ID: <BANLkTikxMk-PGknr08qvvtQzRr-T1eZhcw@mail.gmail.com>
Subject: Re: Site Not Surviving a Single Cassandra Node Crash
From: Vram Kouramajian <vram.kouramajian@gmail.com>
To: user@cassandra.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

The hector clients are used as part of our jetty servers. And, the
jetty servers stop responding when one of the Cassandra nodes go down.

Vram

On Sat, Apr 9, 2011 at 11:54 AM, Joe Stump <joe@joestump.net> wrote:
> Did the Cassandra cluster go down or did you start getting failures from =
the client when it routed queries to the downed node? The key in the client=
 is to keep working around the ring if the initial node is down.
>
> --Joe
>
> On Apr 9, 2011, at 12:52 PM, Vram Kouramajian wrote:
>
>> We have a 5 Cassandra nodes with the following configuration:
>>
>> Casandra Version: 0.6.11
>> Number of Nodes: 5
>> Replication Factor: 3
>> Client: Hector 0.6.0-14
>> Write Consistency Level: Quorum
>> Read Consistency Level: Quorum
>> Ring Topology:
>> =A0 Owns =A0 =A0Range =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 =A0Ring
>>
>> 132756707369141912386052673276321963528
>> 192.168.89.153Up =A0 =A0 =A0 =A0 4.15 GB =A0 =A0 =A0 33.87%
>> 20237398133070283622632741498697119875 =A0 =A0 |<--|
>> 192.168.89.155Up =A0 =A0 =A0 =A0 5.17 GB =A0 =A0 =A0 18.29%
>> 51358066040236348437506517944084891398 =A0 =A0 | =A0 ^
>> 192.168.89.154Up =A0 =A0 =A0 =A0 7.41 GB =A0 =A0 =A0 33.97%
>> 109158969152851862753910401160326064203 =A0 =A0v =A0 |
>> 192.168.89.152Up =A0 =A0 =A0 =A0 5.07 GB =A0 =A0 =A0 6.34%
>> 119944993359936402983569623214763193674 =A0 =A0| =A0 ^
>> 192.168.89.151Up =A0 =A0 =A0 =A0 4.22 GB =A0 =A0 =A0 7.53%
>> 132756707369141912386052673276321963528 =A0 =A0|-->|
>>
>> We believe that our setup should survive the crash of one of the
>> Cassandra nodes. But, we had few crashes and the system stopped
>> functioning until we brought back the Cassandra nodes.
>>
>> Any clues?
>>
>> Vram
>
>