cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jun Rao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CASSANDRA-208) jvm crashes intermittently during compaction
Date Tue, 02 Jun 2009 01:23:07 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715322#action_12715322
] 

Jun Rao commented on CASSANDRA-208:
-----------------------------------

Even if we take out the row index and BF, data is still mixed with column index.

Interestingly, HBase's sstable format started with MapFile (1 file for index, 1 file for BF
and 1 file for data) and has recently moved to the TFile (a spec can be found at https://issues.apache.org/jira/browse/HADOOP-3315)
format (1 file for everything). Although HBase probably did that mainly for reducing # of
files (problem for HDFS master), it's probably worth while to take a look at their new design
(https://issues.apache.org/jira/browse/HBASE-61).

> jvm crashes intermittently during compaction
> --------------------------------------------
>
>                 Key: CASSANDRA-208
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-208
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: trunk
>         Environment: arch: x86_64
> os: Linux version 2.6.18-92.1.22.el5 
> java: nio2-ea-bin-b99-linux-x64-05_feb_2009
>            Reporter: Jiansheng Huang
>            Assignee: Jonathan Ellis
>            Priority: Critical
>             Fix For: 0.3
>
>
> jvm crashes intermittently during compaction. Our test data set is not that big, less
than 10 GB.
> When jvm is about to crash, we see that it consumes a lot of memory (exceeding the max
heap size).
> The excessive memory usage during compaction is caused by the maintenance of blockIndexes_
in SSTable. this blockIndexes_ was only introduced to the apache version.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message