cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Stupp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-7386) JBOD threshold to prevent unbalanced disk utilization
Date Fri, 25 Jul 2014 15:17:38 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074463#comment-14074463
] 

Robert Stupp commented on CASSANDRA-7386:
-----------------------------------------

MBeans give information about the "write value" for each data directory. These should be monitored.
These tests should be performed with and without the patch.

One test is similar to what's reported in CASSANDRA-7615:
# start with half of the disks configured for C* data directories
# add data (using stress tool?)
# after some time, add more data directories
# add more data (using a different keyspace, see below)
# new sstables should prefer the new data directories (they should have a better "write value")
# at some point, new sstables should be distributed equally over all disks and result in approx
same utilization

Next test is to check that heavily utilized (read or write ops) directories are not chosen
for new sstables.
# "Hammer" the first directories (the first keyspace) from the previous test with compactions
or repairs
# Add more data to a new keyspace
# New sstables should go to the 2nd set of data directories.


> JBOD threshold to prevent unbalanced disk utilization
> -----------------------------------------------------
>
>                 Key: CASSANDRA-7386
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7386
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Chris Lohfink
>            Assignee: Robert Stupp
>            Priority: Minor
>         Attachments: 7386-v1.patch, 7386v2.diff, Mappe1.ods, patch_2_1_branch_proto.diff
>
>
> Currently the pick the disks are picked first by number of current tasks, then by free
space.  This helps with performance but can lead to large differences in utilization in some
(unlikely but possible) scenarios.  Ive seen 55% to 10% and heard reports of 90% to 10% on
IRC.  With both LCS and STCS (although my suspicion is that STCS makes it worse since harder
to be balanced).
> I purpose the algorithm change a little to have some maximum range of utilization where
it will pick by free space over load (acknowledging it can be slower).  So if a disk A is
30% full and disk B is 5% full it will never pick A over B until it balances out.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message