Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Message-ID: <25974772.466501285787014748.JavaMail.jira@thor>
Date: Wed, 29 Sep 2010 15:03:34 -0400 (EDT)
From: "Stu Hood (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Subject: [jira] Assigned: (CASSANDRA-1526) Make cassandra sampling and
 startup faster
In-Reply-To: <15972528.321171285090414185.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CASSANDRA-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stu Hood reassigned CASSANDRA-1526:
-----------------------------------

    Assignee: Stu Hood  (was: Jonathan Ellis)

> Make cassandra sampling and startup faster
> ------------------------------------------
>
>                 Key: CASSANDRA-1526
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1526
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Edward Capriolo
>            Assignee: Stu Hood
>            Priority: Minor
>             Fix For: 0.6.7, 0.7.0
>
>         Attachments: 1526.txt, cpu.txt, io.txt, skip-short-byte-array.diff
>
>
> http://wiki.apache.org/cassandra/CassandraHardware makes mention of very large disks I do not see how that would be possible.
> We have a server class system have 4x processors 16GB RAM a 6 DISK RAID5 (yes RAID0 would be faster but still) 
> {noformat}
> INFO [main] 2010-09-21 12:58:26,348 SSTableReader.java (line 120) Sampling index for /var/lib/cassandra/data/system/LocationInfo-699-Data.db
> ...
> INFO [main] 2010-09-21 13:05:51,333 CassandraDaemon.java (line 124) Binding thrift service to cdbsd07/10.71.71.57:9160
> {noformat}
> This node has 200GB of data in two column families and the time to sample all tables and startup is 7+ minutes. The logging suggests this process is happening a single SSTable at a time. Additionally the normal system vitals mainly DISK and CPU do not look overtaxed.
> * Since SSTables are immutable is there a way the sampling of the tables could be saved?
> * Could this process be done in parallel for speedup?
> * Can multiple column families be processed at once?
> Unless someone has an insanely powerful disk pack making mention of 2TB limitations seem out of place. Unless my calculations are wrong (which they usually are), I have a pretty decent hardware, and if I had 2 TB of data I would have a 95 minute node start up? 
> I hope that maybe sampling multiple ColumnFamilies at once would make nodes of at least a few hundred GB startup reasonably fast.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.