cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jun Rao (JIRA)" <>
Subject [jira] Commented: (CASSANDRA-208) jvm crashes intermittently during compaction
Date Tue, 02 Jun 2009 01:23:07 GMT


Jun Rao commented on CASSANDRA-208:

Even if we take out the row index and BF, data is still mixed with column index.

Interestingly, HBase's sstable format started with MapFile (1 file for index, 1 file for BF
and 1 file for data) and has recently moved to the TFile (a spec can be found at
format (1 file for everything). Although HBase probably did that mainly for reducing # of
files (problem for HDFS master), it's probably worth while to take a look at their new design

> jvm crashes intermittently during compaction
> --------------------------------------------
>                 Key: CASSANDRA-208
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: trunk
>         Environment: arch: x86_64
> os: Linux version 2.6.18-92.1.22.el5 
> java: nio2-ea-bin-b99-linux-x64-05_feb_2009
>            Reporter: Jiansheng Huang
>            Assignee: Jonathan Ellis
>            Priority: Critical
>             Fix For: 0.3
> jvm crashes intermittently during compaction. Our test data set is not that big, less
than 10 GB.
> When jvm is about to crash, we see that it consumes a lot of memory (exceeding the max
heap size).
> The excessive memory usage during compaction is caused by the maintenance of blockIndexes_
in SSTable. this blockIndexes_ was only introduced to the apache version.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message