Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
MIME-Version: 1.0
In-Reply-To: <CAD9GGnZ00iGabV9s2iu9N+A=d+soTpzN-RMnjX4UfifNTikEfQ@mail.gmail.com>
References: <CAMy13tDE1D4qtRjJyT+Vv5Nv-zRnWqARoc9NvPoKCS_Jix3o_w@mail.gmail.com>
 <157d35a4996.111bf0c3c419737.893769067162731191@winguzone.com> <CAD9GGnZ00iGabV9s2iu9N+A=d+soTpzN-RMnjX4UfifNTikEfQ@mail.gmail.com>
From: Seth Edwards <seth@pubnub.com>
Date: Mon, 17 Oct 2016 11:11:07 -0700
Message-ID: <CAMy13tA3cZ++LaVnUsuwkwbR5tvBdhMEOqWij9nrWRODq42rLQ@mail.gmail.com>
Subject: Re: Adding disk capacity to a running node
To: user <user@cassandra.apache.org>
Content-Type: multipart/alternative; boundary=047d7bfcf36efbdd15053f137f46
archived-at: Mon, 17 Oct 2016 18:11:14 -0000

--047d7bfcf36efbdd15053f137f46
Content-Type: text/plain; charset=UTF-8

We're running 2.0.16. We're migrating to a new data model but we've had an
unexpected increase in write traffic that has caused us some capacity
issues when we encounter compactions. Our old data model is on STCS. We'd
like to add another ebs volume (we're on aws) to our JBOD config and
hopefully avoid any situation where we run out of disk space during a large
compaction. It appears that the behavior we are hoping to get is actually
undesirable and removed in 3.2. It still might be an option for us until we
can finish the migration.

I'm not familiar with LVM so it may be a bit risky to try at this point.

On Mon, Oct 17, 2016 at 9:42 AM, Yabin Meng <yabinmeng@gmail.com> wrote:

> I assume you're talking about Cassandra JBOD (just a bunch of disk) setup
> because you do mention it as adding it to the list of data directories. If
> this is the case, you may run into issues, depending on your C* version.
> Check this out: http://www.datastax.com/dev/blog/improving-jbod.
>
> Or another approach is to use LVM to manage multiple devices into a single
> mount point. If you do so, from what Cassandra can see is just simply
> increased disk storage space and there should should have no problem.
>
> Hope this helps,
>
> Yabin
>
> On Mon, Oct 17, 2016 at 11:54 AM, Vladimir Yudovin <vladyu@winguzone.com>
> wrote:
>
>> Yes, Cassandra should keep percent of disk usage equal for all disk.
>> Compaction process and SSTable flushes will use new disk to distribute both
>> new and existing data.
>>
>> Best regards, Vladimir Yudovin,
>>
>>
>> *Winguzone <https://winguzone.com?from=list> - Hosted Cloud Cassandra on
>> Azure and SoftLayer.Launch your cluster in minutes.*
>>
>>
>> ---- On Mon, 17 Oct 2016 11:43:27 -0400*Seth Edwards <seth@pubnub.com
>> <seth@pubnub.com>>* wrote ----
>>
>> We have a few nodes that are running out of disk capacity at the moment
>> and instead of adding more nodes to the cluster, we would like to add
>> another disk to the server and add it to the list of data directories. My
>> question, is, will Cassandra use the new disk for compactions on sstables
>> that already exist in the primary directory?
>>
>>
>>
>> Thanks!
>>
>>
>>
>

--047d7bfcf36efbdd15053f137f46
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">We&#39;re running 2.0.16. We&#39;re migrating to a new dat=
a model but we&#39;ve had an unexpected increase in write traffic that has =
caused us some capacity issues when we encounter compactions. Our old data =
model is on STCS. We&#39;d like to add another ebs volume (we&#39;re on aws=
) to our JBOD config and hopefully avoid any situation where we run out of =
disk space during a large compaction. It appears that the behavior we are h=
oping to get is actually undesirable and removed in 3.2. It still might be =
an option for us until we can finish the migration.=C2=A0<div><br></div><di=
v>I&#39;m not familiar with LVM so it may be a bit risky to try at this poi=
nt.=C2=A0</div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quo=
te">On Mon, Oct 17, 2016 at 9:42 AM, Yabin Meng <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:yabinmeng@gmail.com" target=3D"_blank">yabinmeng@gmail.com</a>=
&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0=
 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">I ass=
ume you&#39;re talking about Cassandra JBOD (just a bunch of disk) setup be=
cause you do mention it as adding it to the list of data directories. If th=
is is the case, you may run into issues, depending on your C* version. Chec=
k this out:=C2=A0<a href=3D"http://www.datastax.com/dev/blog/improving-jbod=
" target=3D"_blank">http://www.datastax.com/<wbr>dev/blog/improving-jbod</a=
>.<div><br></div><div>Or another approach is to use LVM to manage multiple =
devices into a single mount point. If you do so, from what Cassandra can se=
e is just simply increased disk storage space and there should should have =
no problem.</div><div><br></div><div>Hope this helps,</div><div><br></div><=
div>Yabin</div></div><div class=3D"HOEnZb"><div class=3D"h5"><div class=3D"=
gmail_extra"><br><div class=3D"gmail_quote">On Mon, Oct 17, 2016 at 11:54 A=
M, Vladimir Yudovin <span dir=3D"ltr">&lt;<a href=3D"mailto:vladyu@winguzon=
e.com" target=3D"_blank">vladyu@winguzone.com</a>&gt;</span> wrote:<br><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c=
cc solid;padding-left:1ex"><u></u><div><div style=3D"font-size:10pt;font-fa=
mily:Verdana,Arial,Helvetica,sans-serif"><div>Yes, Cassandra should keep pe=
rcent of disk usage equal for all disk. Compaction process and SSTable flus=
hes will use new disk to distribute both new and existing data.<br></div><d=
iv><br></div><div id=3D"m_-1028774157634255612m_-4473064825674527426"><div>=
Best regards, Vladimir Yudovin, <br></div><div><i><a href=3D"https://winguz=
one.com?from=3Dlist" target=3D"_blank">Winguzone</a> - Hosted Cloud Cassand=
ra on Azure and SoftLayer.<br>Launch your cluster in minutes.<br></i></div>=
</div><div><br></div><div class=3D"m_-1028774157634255612m_-447306482567452=
7426zmail_extra"><div id=3D"m_-1028774157634255612m_-44730648256745274261">=
<div><br></div><div>---- On Mon, 17 Oct 2016 11:43:27 -0400<b>Seth Edwards =
&lt;<a href=3D"mailto:seth@pubnub.com" target=3D"_blank">seth@pubnub.com</a=
>&gt;</b> wrote ----<br></div></div><span><div><br></div><blockquote style=
=3D"border-left:1px solid #cccccc;padding-left:6px;margin:0 0 0 5px"><div><=
div dir=3D"ltr"><div>We have a few nodes that are running out of disk capac=
ity at the moment and instead of adding more nodes to the cluster, we would=
 like to add another disk to the server and add it to the list of data dire=
ctories. My question, is, will Cassandra use the new disk for compactions o=
n sstables that already exist in the primary directory?=C2=A0<br></div><div=
><br></div><div><br></div><div><br></div><div>Thanks!<br></div></div></div>=
</blockquote></span></div><div><br></div></div></div></blockquote></div><br=
></div>
</div></div></blockquote></div><br></div>

--047d7bfcf36efbdd15053f137f46--