incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Lebresne <sylv...@datastax.com>
Subject Re: OOM on CompressionMetadata.readChunkOffsets(..)
Date Mon, 31 Oct 2011 14:08:21 GMT
On Mon, Oct 31, 2011 at 2:58 PM, Sylvain Lebresne <sylvain@datastax.com> wrote:
> On Mon, Oct 31, 2011 at 1:10 PM, Mick Semb Wever <mck@apache.org> wrote:
>> On Mon, 2011-10-31 at 13:05 +0100, Mick Semb Wever wrote:
>>> Given a 60G sstable, even with 64kb chunk_length, to read just that one
>>> sstable requires close to 8G free heap memory...
>>
>> Arg, that calculation was a little off...
>>  (a long isn't exactly 8K...)
>>
>> But you get my concern...
>
> Well, with a long being only 8 bytes, that's 8MB of free heap memory. Without
> being negligible, that's not completely crazy to me.
>
> No, the problem is that we create those 8MB for each reads, which *is* crazy
> (the fact that we allocate those 8MB in one block is not very nice for
> the GC either
> but that's another problem).
> Anyway, that's really a bug and I've created CASSANDRA-3427 to fix.

Note that it's only a problem for range queries.

--
Sylvain

>
> --
> Sylvain
>
>>
>> ~mck
>>
>> --
>> "When you say: "I wrote a program that crashed Windows", people just
>> stare at you blankly and say: "Hey, I got those with the system -- for
>> free."" Linus Torvalds
>>
>> | http://semb.wever.org | http://sesat.no |
>> | http://tech.finn.no   | Java XSS Filter |
>>
>

Mime
View raw message