cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Or Sher <or.sh...@gmail.com>
Subject Re: Replacing nodes disks
Date Sun, 21 Dec 2014 12:01:00 GMT
If I'll use the replace_address parameter with the same IP address, would
that do the job?

On Sun, Dec 21, 2014 at 11:20 AM, Or Sher <or.sher1@gmail.com> wrote:

> What I want to do is kind of replacing a dead node -
> http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_replace_node_t.html
> But replacing it with a clean node with the same IP and hostname.
>
> On Sun, Dec 21, 2014 at 9:53 AM, Or Sher <or.sher1@gmail.com> wrote:
>
>> Thanks guys.
>> I have to replace all data disks, so I don't have another large enough
>> local disk to move the data to.
>> If I'll have no choice, I will backup the data before on some other node
>> or something, but I'd like to avoid it.
>> I would really love letting Cassandra do it thing and rebuild itself.
>> Did anybody handled such cases that way (Letting Cassandra rebuild it's
>> data?)
>> Although there are no documented procedure for it, It should be possible
>> right?
>>
>> On Fri, Dec 19, 2014 at 8:41 AM, Jan Kesten <j.kesten@enercast.de> wrote:
>>
>>> Hi Or,
>>>
>>> I did some sort of this a while ago. If your machines do have a free
>>> disk slot - just put another disk there and use it as another
>>> data_file_directory.
>>>
>>> If not - as in my case:
>>>
>>> - grab an usb dock for disks
>>> - put the new one in there, plug in, format, mount to /mnt etc.
>>> - I did an online rsync from /var/lib/cassandra/data to /mnt
>>> - after that, bring cassandra down
>>> - do another rsync from /var/lib/cassandra/data to /mnt (should be
>>> faster, as sstables do not change, minimizes downtime)
>>> - if you need adjust /etc/fstab if needed
>>> - shutdown the node
>>> - swap disks
>>> - power on the node
>>> - everything should be fine ;-)
>>>
>>> Of course you will need a replication factor > 1 for this to work ;-)
>>>
>>> Just my 2 cents,
>>> Jan
>>>
>>> rsync the full contents there,
>>>
>>> Am 18.12.2014 um 16:17 schrieb Or Sher:
>>>
>>>  Hi all,
>>>>
>>>> We have a situation where some of our nodes have smaller disks and we
>>>> would like to align all nodes by replacing the smaller disks to bigger ones
>>>> without replacing nodes.
>>>> We don't have enough space to put data on / disk and copy it back to
>>>> the bigger disks so we would like to rebuild the nodes data from other
>>>> replicas.
>>>>
>>>> What do you think should be the procedure here?
>>>>
>>>> I'm guessing it should be something like this but I'm pretty sure it's
>>>> not enough.
>>>> 1. shutdown C* node and server.
>>>> 2. replace disks + create the same vg lv etc.
>>>> 3. start C* (Normally?)
>>>> 4. nodetool repair/rebuild?
>>>> *I think I might get some consistency issues for use cases relying on
>>>> Quorum reads and writes for strong consistency.
>>>> What do you say?
>>>>
>>>> Another question is (and I know it depends on many factors but I'd like
>>>> to hear an experienced estimation): How much time would take to rebuild a
>>>> 250G data node?
>>>>
>>>> Thanks in advance,
>>>> Or.
>>>>
>>>> --
>>>> Or Sher
>>>>
>>>
>>>
>>
>>
>> --
>> Or Sher
>>
>
>
>
> --
> Or Sher
>



-- 
Or Sher

Mime
View raw message