cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Lebresne <sylv...@datastax.com>
Subject Re: OOM on CompressionMetadata.readChunkOffsets(..)
Date Mon, 31 Oct 2011 13:58:20 GMT
On Mon, Oct 31, 2011 at 1:10 PM, Mick Semb Wever <mck@apache.org> wrote:
> On Mon, 2011-10-31 at 13:05 +0100, Mick Semb Wever wrote:
>> Given a 60G sstable, even with 64kb chunk_length, to read just that one
>> sstable requires close to 8G free heap memory...
>
> Arg, that calculation was a little off...
>  (a long isn't exactly 8K...)
>
> But you get my concern...

Well, with a long being only 8 bytes, that's 8MB of free heap memory. Without
being negligible, that's not completely crazy to me.

No, the problem is that we create those 8MB for each reads, which *is* crazy
(the fact that we allocate those 8MB in one block is not very nice for
the GC either
but that's another problem).
Anyway, that's really a bug and I've created CASSANDRA-3427 to fix.

--
Sylvain

>
> ~mck
>
> --
> "When you say: "I wrote a program that crashed Windows", people just
> stare at you blankly and say: "Hey, I got those with the system -- for
> free."" Linus Torvalds
>
> | http://semb.wever.org | http://sesat.no |
> | http://tech.finn.no   | Java XSS Filter |
>

Mime
View raw message