cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vincent gromakowski <vincent.gromakow...@gmail.com>
Subject Re: Using K8s to Manage Cassandra in Production
Date Wed, 23 May 2018 14:04:31 GMT
Why did you choose java for the operator implementation when everybody
seems to use the go client (probably for greater functionalities) ?

2018-05-23 15:39 GMT+02:00 Ben Bromhead <ben@instaclustr.com>:

> You can get a good way with StatefulSets, but as Tom mentioned there are
> still some issues with this, particularly around scaling up and down.
>
> We are working on an Operator for Apache Cassandra, you can find it here
> https://github.com/instaclustr/cassandra-operator. This is a joint
> project between Instaclustr, Pivotal and a few other folk.
>
> Currently it's a work in progress, but we would love any or all early
> feedback/PRs/issues etc. Our first GA release will target the following
> capabilities:
>
>    - Safe scaling up and down (including decommissioning)
>    - Backup/restore workflow (snapshots only initially)
>    - Built in prometheus integration and discovery
>
> Other features like repair, better PV support, maybe even a nice dashboard
> will be on the way.
>
>
> On Wed, May 23, 2018 at 7:35 AM Tom Petracca <tpetracca@palantir.com>
> wrote:
>
>> Using a statefulset should get you pretty far, though will likely be less
>> effective than a coreos-style “operator”. Some random points:
>>
>>    - For scale-up: a node shouldn’t report “ready” until it’s in the
>>    NORMAL state; this will prevent multiple nodes from bootstrapping at once.
>>    - For scale-down: as of now there isn’t a mechanism to know if a pod
>>    is getting decommissioned because you’ve permanently lowered replica count,
>>    or because it’s just getting bounced/re-scheduled, thus knowing whether or
>>    not to decommission is basically impossible. Relevant issue:
>>    kubernetes/kubernetes#1462
>>    <https://github.com/kubernetes/kubernetes/issues/1462>
>>
>>
>>
>> *From: *Pradeep Chhetri <pradeep@stashaway.com>
>> *Reply-To: *"user@cassandra.apache.org" <user@cassandra.apache.org>
>> *Date: *Friday, May 18, 2018 at 10:20 AM
>> *To: *"user@cassandra.apache.org" <user@cassandra.apache.org>
>> *Subject: *Re: Using K8s to Manage Cassandra in Production
>>
>>
>>
>> Hello Hassaan,
>>
>>
>>
>> We use cassandra helm chart[0] for deploying cassandra over kubernetes in
>> production. We have around 200GB cas data. It works really well. You can
>> scale up nodes easily (I haven't tested scaling down).
>>
>>
>>
>> I would say that if you are worried about running cassandra over k8s in
>> production, maybe you should first try setting it for your
>> staging/preproduction and gain confidence over time.
>>
>>
>>
>> I have tested situations where i have killed the host running cassandra
>> container and have seen that container moves to a different node and joins
>> cluster properly. So from my experience its pretty good. No issues till yet.
>>
>>
>>
>> [0]: https://github.com/kubernetes/charts/tree/master/incubator/cassandra
>> [github.com]
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_charts_tree_master_incubator_cassandra&d=DwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=1oh1YI8i5eJD1DFTwooO7U92fFi2fjan6lqP61yAiyo&m=dupKDpZi0lkjFkqaSd6XaEj5nuY1T5UObgAcXCBqo7A&s=0WTYStEM1zvh2BQKvnVLRpukxgr0aDLyGffyE1V2xik&e=>
>>
>>
>>
>>
>>
>> Regards,
>>
>> Pradeep
>>
>>
>>
>> On Fri, May 18, 2018 at 1:01 PM, Павел Сапежко <amelius0712@gmail.com>
>> wrote:
>>
>> Hi, Hassaan! For example we are using C* in k8s in production for our
>> video surveillance system. Moreover, we are using Ceph RBD as our storage
>> for cassandra. Today we have 8 C* nodes each manages 2Tb of data.
>>
>>
>>
>> On Fri, May 18, 2018 at 9:27 AM Hassaan Pasha <hpasha@an10.io> wrote:
>>
>> Hi,
>>
>>
>>
>> I am trying to craft a deployment strategy for deploying and maintaining
>> a C* cluster. I was wondering if there are actual production deployments of
>> C* using K8s as the orchestration layer.
>>
>>
>>
>> I have been given the impression that K8s managing a C* cluster can be a
>> recipe for disaster, especially if you aren't well versed with the
>> intricacies of a scale-up/down event. I know use cases where people are
>> using Mesos or a custom tool built with terraform/chef etc to run their
>> production clusters but have yet to find a real K8s use case.
>>
>>
>>
>> *Questions?*
>>
>> Is K8s a reasonable choice for managing a production C* cluster?
>>
>> Are there documented use cases for this?
>>
>>
>>
>> Any help would be greatly appreciated.
>>
>>
>>
>> --
>>
>> Regards,
>>
>>
>>
>> *Hassaan Pasha*
>>
>> --
>>
>> Regrads,
>>
>> Pavel Sapezhko
>>
>>
>>
> --
> Ben Bromhead
> CTO | Instaclustr <https://www.instaclustr.com/>
> +1 650 284 9692
> Reliability at Scale
> Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
>

Mime
View raw message