cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Burress <philburress...@gmail.com>
Subject Re: Bootstrap Timing
Date Sun, 20 Apr 2014 01:19:51 GMT
Thank you all for your advice and good info. The node has died a couple of
times with out of memory errors. I've restarted each time but it starts re
- running compaction and then dies again.

Is there a better way to do this?
On Apr 18, 2014 6:06 PM, "Steven A Robenalt" <srobenal@stanford.edu> wrote:

> That's what I'd be doing, but I wouldn't expect it to run for 3 days this
> time. My guess is that whatever was going wrong with the bootstrap when you
> had 3 nodes starting at once was interfering with the completion of the 1
> remaining node of those 3. A clean bootstrap of a single node should
> complete eventually, and I would think it'll be a lot less than 3 days. Our
> database is much smaller than yours at the moment, so I can't really guide
> you on how long it should take, but I'd think that others on the list with
> similar database sizes might be able to give you a better idea.
>
> Steve
>
>
>
> On Fri, Apr 18, 2014 at 1:43 PM, Phil Burress <philburresseme@gmail.com>wrote:
>
>> First, I just stopped 2 of the nodes and left one running. But this
>> morning, I stopped that third node, cleared out the data, restarted and let
>> it rejoin again. It appears streaming is done (according to netstats),
>> right now it appears to be running compaction and building secondary index
>> (according to compactionstats). Just sit and wait I guess?
>>
>>
>> On Fri, Apr 18, 2014 at 2:23 PM, Steven A Robenalt <srobenal@stanford.edu
>> > wrote:
>>
>>> Looking back through this email chain, it looks like Phil said he wasn't
>>> using vnodes.
>>>
>>> For the record, we are using vnodes since we brought up our first
>>> cluster, and have not seen any issues with bootstrapping new nodes either
>>> to replace existing nodes, or to grow/shrink the cluster. We did adhere to
>>> the caveats that new nodes should not be seed nodes, and that we should
>>> allow each node to join the cluster completely before making any other
>>> changes.
>>>
>>> Phil, when you dropped to adding just the single node to your cluster,
>>> did you start over with the newly added node (blowing away the database
>>> created on the previous startup), or did you shut down the other 2 added
>>> nodes and leave the remaining one in progress to continue?
>>>
>>> Steve
>>>
>>>
>>> On Fri, Apr 18, 2014 at 10:40 AM, Robert Coli <rcoli@eventbrite.com>wrote:
>>>
>>>> On Fri, Apr 18, 2014 at 5:05 AM, Phil Burress <philburresseme@gmail.com
>>>> > wrote:
>>>>
>>>>> nodetool netstats shows 84 files. They are all at 100%. Nothing
>>>>> showing in Pending or Active for Read Repair Stats.
>>>>>
>>>>> I'm assuming this means it's done. But it still shows "JOINING". Is
>>>>> there an undocumented step I'm missing here? This whole process seems
>>>>> broken to me.
>>>>>
>>>>
>>>> Lately it seems like a lot more people than usual are :
>>>>
>>>> 1) using vnodes
>>>> 2) unable to bootstrap new nodes
>>>>
>>>> If I were you, I would likely file a JIRA detailing your negative
>>>> experience with this core functionality.
>>>>
>>>> =Rob
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Steve Robenalt
>>> Software Architect
>>>  HighWire | Stanford University
>>> 425 Broadway St, Redwood City, CA 94063
>>>
>>> srobenal@stanford.edu
>>> http://highwire.stanford.edu
>>>
>>>
>>>
>>>
>>>
>>>
>>
>
>
> --
> Steve Robenalt
> Software Architect
> HighWire | Stanford University
> 425 Broadway St, Redwood City, CA 94063
>
> srobenal@stanford.edu
> http://highwire.stanford.edu
>
>
>
>
>
>

Mime
View raw message