cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Jirsa <jji...@gmail.com>
Subject Re: Cassandra limitations
Date Fri, 04 May 2018 18:24:38 GMT
Cluster. The overhead is per cluster.

There are two places you'll run into scaling pain here.

1) Size of the schema (which we have to serialize to send around) - too
many tables, or too many columns in tables, can cause serializing schema to
get really expensive and cause problems
2) Too many memtables - assume that all of them will have some tiny trivial
amount of data in them, maybe 1MB. 200 * 1MB = 200MB of heap just for empty
memtables. If you have a thousand tables, that's a gigabyte of heap, just
for EMPTY memtables.





On Fri, May 4, 2018 at 11:17 AM, Abdul Patel <abd786.ap@gmail.com> wrote:

> I have 3 projects in pipeline adding 3 different cluster  across all
> environwments would too costly option :)
>
> So 200 tables per keyspace or per cluster?
>
>
> On Friday, May 4, 2018, Durity, Sean R <SEAN_R_DURITY@homedepot.com>
> wrote:
>
>> The issue is more with the number of tables, not the number of keyspaces.
>> Because each table has a memTable, there is a practical limit to the number
>> of memtables that a node can hold in its memory. (And scaling out doesn’t
>> help, because every node still has a memTable for every table.) The
>> practical table limit I have heard is in the low hundreds – maybe 200 as a
>> rough estimate.
>>
>>
>>
>> In general, we create a new cluster (instead of a new keyspace) for each
>> application.
>>
>>
>>
>>
>>
>> Sean Durity
>>
>> *From:* Abdul Patel <abd786.ap@gmail.com>
>> *Sent:* Thursday, May 03, 2018 5:56 PM
>> *To:* User@cassandra.apache.org
>> *Subject:* [EXTERNAL] Cassandra limitations
>>
>>
>>
>> Hi ,
>>
>>
>>
>> In my environment, we are coming up with 3 to 4 new projects , hence new
>> keyspaces will be coming into picture.
>>
>> Do we have any limitations or performance issues when we hit to a number
>> of keyspaces or number of nodes vs keyspaces?
>>
>> Also connections limitations if any?
>>
>>
>>
>> I know as data grows we can add more nodes and memory but nor sure about
>> somethinh else which need to take into consideration.
>>
>>
>>
>>
>>
>> ------------------------------
>>
>> The information in this Internet Email is confidential and may be legally
>> privileged. It is intended solely for the addressee. Access to this Email
>> by anyone else is unauthorized. If you are not the intended recipient, any
>> disclosure, copying, distribution or any action taken or omitted to be
>> taken in reliance on it, is prohibited and may be unlawful. When addressed
>> to our clients any opinions or advice contained in this Email are subject
>> to the terms and conditions expressed in any applicable governing The Home
>> Depot terms of business or client engagement letter. The Home Depot
>> disclaims all responsibility and liability for the accuracy and content of
>> this attachment and for any damages or losses arising from any
>> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
>> items of a destructive nature, which may be contained in this attachment
>> and shall not be liable for direct, indirect, consequential or special
>> damages in connection with this e-mail message or its attachment.
>>
>

Mime
View raw message