Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Tue, 16 Dec 2014 21:54:13 +0000 (UTC)
From: "Ryan Svihla (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12762083.1418758459000.38158.1418766853771@Atlassian.JIRA>
In-Reply-To: <JIRA.12762083.1418758459000@Atlassian.JIRA>
References: <JIRA.12762083.1418758459000@Atlassian.JIRA>
 <JIRA.12762083.1418758459840@arcas>
Subject: [jira] [Commented] (CASSANDRA-8494) incremental bootstrap
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-8494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249029#comment-14249029 ] 

Ryan Svihla commented on CASSANDRA-8494:
----------------------------------------

I'm mixed on the push for density, I get that people _really_ want it, and this would substantially help Cassandra in that space, but I'm also convinced just by physics the story for high density will always be worse than the story for a bunch of cheap low density nodes (IE total cost, not just data center space costs).

Regardless, i think even in the case of more say 1TB nodes, this would be an impressive boost to handling overloaded clusters, where load can be moved off struggling nodes more quickly and gracefully. What we struggle with today in the field is a people that don't monitor their clusters, and don't realize till they're going OOM that they're in trouble. For those folks we always struggle streaming in new nodes as quickly as possible. I think this could potentially really help with those more common than you'd think scenarios.

> incremental bootstrap
> ---------------------
>
>                 Key: CASSANDRA-8494
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8494
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Jon Haddad
>            Priority: Minor
>
> Current bootstrapping involves (to my knowledge) picking tokens and streaming data before the node is available for requests.  This can be problematic with "fat nodes", since it may require 20TB of data to be streamed over before the machine can be useful.  This can result in a massive window of time before the machine can do anything useful.
> As a potential approach to mitigate the huge window of time before a node is available, I suggest modifying the bootstrap process to only acquire a single initial token before being marked UP.  This would likely be a configuration parameter "incremental_bootstrap" or something similar.
> After the node is bootstrapped with this one token, it could go into UP state, and could then acquire additional tokens (one or a handful at a time), which would be streamed over while the node is active and serving requests.  The benefit here is that with the default 256 tokens a node could become an active part of the cluster with less than 1% of it's final data streamed over.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)