cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Plotnik, Alexey" <aplot...@rhonda.ru>
Subject RE: Turn off compression (1.2.11)
Date Mon, 24 Feb 2014 01:32:11 GMT
It’s not wrong in case of economy, we have many TB of data, and it’s very expensive to
have even 3TB per machine (we need 10TB minimum I think).
The main thing you should understand – TB is not a problem, the problem is how many rows
you have per node.

From: Yogi Nerella [mailto:ynerella999@gmail.com]
Sent: 19 февраля 2014 г. 10:21
To: user@cassandra.apache.org
Subject: Re: Turn off compression (1.2.11)

I am new and trying to learn Cassandra.

Based on my understanding of the problem, almost 2Gb is taken up just for the compression
headers heap.

And 100MB per SSTable, and about 30,000 files gives about 3TB of data?

What is the hardware and memory configuration you are using to provide this large data?

Should this be reduced to smaller data sets, and partitioned into multiple nodes?

If my understanding is  totally wrong, please forgive and if possible explain.






On Tue, Feb 18, 2014 at 2:58 PM, Plotnik, Alexey <aplotnik@rhonda.ru<mailto:aplotnik@rhonda.ru>>
wrote:
Compression buffers are located in Heap, I saw them in Heapdump. That is:

======================
public class CompressedRandomAccessReader extends RandomAccessReader {
…..
   private ByteBuffer compressed; // <-- THAT IS
======================

From: Robert Coli [mailto:rcoli@eventbrite.com<mailto:rcoli@eventbrite.com>]
Sent: 19 февраля 2014 г. 6:24
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Turn off compression (1.2.11)

On Mon, Feb 17, 2014 at 4:35 PM, Plotnik, Alexey <aplotnik@rhonda.ru<mailto:aplotnik@rhonda.ru>>
wrote:

As an aside, 1.2.0 beta moved a bunch of data related to compression off the heap. If you
were to try to run the same cluster under 1.1, you'd probably OOM your heap immediately.

https://issues.apache.org/jira/browse/CASSANDRA-4941

=Rob


Mime
View raw message