Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: error (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CAG2rwuN78E3-WUM-+SS3qbGaumTPxbYYsQwKXcZn+kojX_Ubag@mail.gmail.com>
References: 
 <CAG2rwuOpBWu1wLDYHFZY2Q084fW561_t=pMMcZy7BsicBpdqsQ@mail.gmail.com>
	<CAKv2g8eqrFFPCqARxF6BtegYM7fGKEFb_5cectf9DJvRoyvxhw@mail.gmail.com>
	<CAG2rwuN78E3-WUM-+SS3qbGaumTPxbYYsQwKXcZn+kojX_Ubag@mail.gmail.com>
Date: Tue, 9 Jul 2013 09:18:16 +0200
Message-ID: 
 <CAMR1f-ekbu8veKAYQ-700kYyTwCpH9iCmkjaLQ2ZxkcnL5TLog@mail.gmail.com>
Subject: Re: does anyone store large values in cassandra e.g. 100kb?
From: Theo Hultberg <theo@iconara.net>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=047d7b342e18f9ad0504e10ef69a

--047d7b342e18f9ad0504e10ef69a
Content-Type: text/plain; charset=ISO-8859-1

We store objects that are a couple of tens of K, sometimes 100K, and we
store quite a few of these per row, sometimes hundreds of thousands.

One problem we encountered early was that these rows would become so big
that C* couldn't compact the rows in-memory and had to revert to slow
two-pass compactions where it spills partially compacted rows to disk. we
solved that in two ways, first by
increasing in_memory_compaction_limit_in_mb from 64 to 128, and although it
helped a little bit we quickly realized didn't have much effect because
most of the time was taken up by really huge rows many times larger than
that.

We ended up implementing a simple sharding scheme where each row is
actually 36 rows that each contain 1/36 of the range (we take the first
letter in the column key and stick that on the row key on writes, and on
reads we read all 36 rows -- 36 because there are 36 letters and numbers in
the ascii alphabet and our column keys happen to distribute over that quite
nicely).

Cassandra works well with semi-large objects, and it works well with wide
rows, but you have to be careful about the combination where rows get
larger than 64 Mb.

T#


On Mon, Jul 8, 2013 at 8:13 PM, S Ahmed <sahmed1020@gmail.com> wrote:

> Hi Peter,
>
> Can you describe your environment, # of documents and what kind of usage
> pattern you have?
>
>
>
>
> On Mon, Jul 8, 2013 at 2:06 PM, Peter Lin <woolfel@gmail.com> wrote:
>
>> I regularly store word and pdf docs in cassandra without any issues.
>>
>>
>>
>>
>> On Mon, Jul 8, 2013 at 1:46 PM, S Ahmed <sahmed1020@gmail.com> wrote:
>>
>>> I'm guessing that most people use cassandra to store relatively smaller
>>> payloads like 1-5kb in size.
>>>
>>> Is there anyone using it to store say 100kb (1/10 of a megabyte) and if
>>> so, was there any tweaking or gotchas that you ran into?
>>>
>>
>>
>

--047d7b342e18f9ad0504e10ef69a
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">We store objects that are a couple of tens of K, sometimes=
 100K, and we store quite a few of these per row, sometimes hundreds of tho=
usands.<div><br></div><div>One problem we encountered early was that these =
rows would become so big that C* couldn&#39;t compact the rows in-memory an=
d had to revert to slow two-pass compactions where it spills partially comp=
acted rows to disk. we solved that in two ways, first by increasing=A0in_me=
mory_compaction_limit_in_mb from 64 to 128, and although it helped a little=
 bit we quickly realized didn&#39;t have much effect because most of the ti=
me was taken up by really huge rows many times larger than that.</div>
<div><br></div><div>We ended up implementing a simple sharding scheme where=
 each row is actually 36 rows that each contain 1/36 of the range (we take =
the first letter in the column key and stick that on the row key on writes,=
 and on reads we read all 36 rows -- 36 because there are 36 letters and nu=
mbers in the ascii alphabet and our column keys happen to distribute over t=
hat quite nicely).</div>
<div><br></div><div>Cassandra works well with semi-large objects, and it wo=
rks well with wide rows, but you have to be careful about the combination w=
here rows get larger than 64 Mb.<br><div><br></div><div>T#</div></div></div=
>
<div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Mon, Jul 8=
, 2013 at 8:13 PM, S Ahmed <span dir=3D"ltr">&lt;<a href=3D"mailto:sahmed10=
20@gmail.com" target=3D"_blank">sahmed1020@gmail.com</a>&gt;</span> wrote:<=
br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left=
:1px #ccc solid;padding-left:1ex">
<div dir=3D"ltr">Hi Peter,<div><br></div><div>Can you describe your environ=
ment, # of documents and what kind of usage pattern you have?</div><div><br=
></div><div><br></div></div><div class=3D"HOEnZb"><div class=3D"h5"><div cl=
ass=3D"gmail_extra">
<br>

<br><div class=3D"gmail_quote">On Mon, Jul 8, 2013 at 2:06 PM, Peter Lin <s=
pan dir=3D"ltr">&lt;<a href=3D"mailto:woolfel@gmail.com" target=3D"_blank">=
woolfel@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote=
" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">


<div dir=3D"ltr"><div>I regularly store word and pdf docs in cassandra with=
out any issues.<br><br></div><br></div><div><div><div class=3D"gmail_extra"=
><br><br><div class=3D"gmail_quote">On Mon, Jul 8, 2013 at 1:46 PM, S Ahmed=
 <span dir=3D"ltr">&lt;<a href=3D"mailto:sahmed1020@gmail.com" target=3D"_b=
lank">sahmed1020@gmail.com</a>&gt;</span> wrote:<br>


<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">I&#39;m guessing that most =
people use cassandra to store relatively smaller payloads like 1-5kb in siz=
e.<div>


<br></div><div>Is there anyone using it to store say 100kb (1/10 of a megab=
yte) and if so, was there any tweaking or gotchas that you ran into?</div>

</div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--047d7b342e18f9ad0504e10ef69a--