Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAOxAL62vhTXcrDyD4oVb53xYyE4OktN=LDFrcUuhP6nsNbzW3Q@mail.gmail.com>
References: 
 <CAAZU44nO=65-oStigDsyZodVA+8Q2kUV8LPQmx_BXPje0gC5GQ@mail.gmail.com>
 <CAOxAL62vhTXcrDyD4oVb53xYyE4OktN=LDFrcUuhP6nsNbzW3Q@mail.gmail.com>
From: DuyHai Doan <doanduyhai@gmail.com>
Date: Tue, 19 Jan 2016 09:34:47 +0100
Message-ID: 
 <CABNXB2BNkh6rMzcnrTRS_wjn70oENp6Qx+v-tXwmM5k1Mk=Vag@mail.gmail.com>
Subject: Re: Using cassandra a BLOB store / web cache.
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=001a114e68f026cc630529abbf21

--001a114e68f026cc630529abbf21
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

There is also an excellent tutorial video done by Patrick McFadin and Aaron
Morton on the subject of data model for storing images into Cassandra:
http://youtu.be/gk-B75xgFUg

I guess it can be adapted to store binary objects other than images

On Tue, Jan 19, 2016 at 6:37 AM, Jack Krupansky <jack.krupansky@gmail.com>
wrote:

> Chunk the blobs and store them in a separate table from the metadata.
>
> Here's an old attempt at a chunked object store, for reference:
> https://github.com/Netflix/astyanax/wiki/Chunked-Object-Store
>
> Picking an appropriate chunk size may be key (or not). Somewhere between
> 8K and 512K, I would guess, but it probably doesn't matter a lot and coul=
d
> be tuned and even configured dynamically.
> With a smaller chunk size you have the option of reading lots of small
> chunks with separate requests or as a slice.
>
> It seems like there has been a fair amount of negative sentiment about
> using Cassandra as a blob/object store, but I personally do think it is
> workable to at least some extent.
>
> A lot of my background is with Solr for search, including a little with
> DSE Search.
>
>
> -- Jack Krupansky
>
> On Mon, Jan 18, 2016 at 9:52 PM, Kevin Burton <burton@spinn3r.com> wrote:
>
>> Internally we have the need for a blob store for web content.  It's
>> MOSTLY key, ,value based but we'd like to have lookups by coarse grained
>> tags.
>>
>> This needs to store normal web content like HTML , CSS, JPEG, SVG, etc.
>>
>> Highly doubt that anything over 5MB would need to be stored.
>>
>> We also need the ability to store older versions of the same URL for
>> features like "time travel" where we can see what the web looks like ove=
r
>> time.
>>
>> I initially wrote this for Elasticsearch (and it works well for that) bu=
t
>> it looks like binaries snuck into the set of requirements.
>>
>> I could Base64 encode/decode them in ES I guess but that seems ugly.
>>
>> I was thinking of porting this over to CS but I'm not up to date on the
>> current state of blobs in C*...
>>
>> Any advice?
>>
>> --
>>
>> We=E2=80=99re hiring if you know of any awesome Java Devops or Linux Ope=
rations
>> Engineers!
>>
>> Founder/CEO Spinn3r.com
>> Location: *San Francisco, CA*
>> blog: http://burtonator.wordpress.com
>> =E2=80=A6 or check out my Google+ profile
>> <https://plus.google.com/102718274791889610666/posts>
>>
>>
>

--001a114e68f026cc630529abbf21
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">There is also an excellent tutorial video done by Patrick =
McFadin and Aaron Morton on the subject of data model for storing images in=
to Cassandra:=C2=A0<a href=3D"http://youtu.be/gk-B75xgFUg">http://youtu.be/=
gk-B75xgFUg</a><div><br></div><div>I guess it can be adapted to store binar=
y objects other than images</div></div><div class=3D"gmail_extra"><br><div =
class=3D"gmail_quote">On Tue, Jan 19, 2016 at 6:37 AM, Jack Krupansky <span=
 dir=3D"ltr">&lt;<a href=3D"mailto:jack.krupansky@gmail.com" target=3D"_bla=
nk">jack.krupansky@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"=
gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-=
left:1ex"><div dir=3D"ltr">Chunk the blobs and store them in a separate tab=
le from the metadata.<div><br></div><div>Here&#39;s an old attempt at a chu=
nked object store, for reference:</div><div><a href=3D"https://github.com/N=
etflix/astyanax/wiki/Chunked-Object-Store" target=3D"_blank">https://github=
.com/Netflix/astyanax/wiki/Chunked-Object-Store</a><br></div><div><br></div=
><div>Picking an appropriate chunk size may be key (or not). Somewhere betw=
een 8K and 512K, I would guess, but it probably doesn&#39;t matter a lot an=
d could be tuned and even configured dynamically.</div><div>With a smaller =
chunk size you have the option of reading lots of small chunks with separat=
e requests or as a slice.</div><div><br></div><div>It seems like there has =
been a fair amount of negative sentiment about using Cassandra as a blob/ob=
ject store, but I personally do think it is workable to at least some exten=
t.</div><div><br></div><div>A lot of my background is with Solr for search,=
 including a little with DSE Search.</div><span class=3D"HOEnZb"><font colo=
r=3D"#888888"><div><br></div></font></span></div><div class=3D"gmail_extra"=
><span class=3D"HOEnZb"><font color=3D"#888888"><br clear=3D"all"><div><div=
><div dir=3D"ltr">-- Jack Krupansky</div></div></div></font></span><div><di=
v class=3D"h5">
<br><div class=3D"gmail_quote">On Mon, Jan 18, 2016 at 9:52 PM, Kevin Burto=
n <span dir=3D"ltr">&lt;<a href=3D"mailto:burton@spinn3r.com" target=3D"_bl=
ank">burton@spinn3r.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail=
_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:=
1ex"><div dir=3D"ltr">Internally we have the need for a blob store for web =
content.=C2=A0 It&#39;s MOSTLY key, ,value based but we&#39;d like to have =
lookups by coarse grained tags.<div><br></div><div>This needs to store norm=
al web content like HTML , CSS, JPEG, SVG, etc.</div><div><br></div><div>Hi=
ghly doubt that anything over 5MB would need to be stored.</div><div><br></=
div><div>We also need the ability to store older versions of the same URL f=
or features like &quot;time travel&quot; where we can see what the web look=
s like over time.</div><div><br></div><div>I initially wrote this for Elast=
icsearch (and it works well for that) but it looks like binaries snuck into=
 the set of requirements. =C2=A0</div><div><br></div><div>I could Base64 en=
code/decode them in ES I guess but that seems ugly. =C2=A0</div><div><br></=
div><div>I was thinking of porting this over to CS but I&#39;m not up to da=
te on the current state of blobs in C*...</div><div><br></div><div>Any advi=
ce?=C2=A0<span><font color=3D"#888888"><br></font></span></div><span><font =
color=3D"#888888"><div><div><br></div>-- <br><div><div dir=3D"ltr"><div><di=
v dir=3D"ltr"><div><div dir=3D"ltr"><div><div><p style=3D"margin-top:0px;ma=
rgin-right:0px;margin-bottom:12pt;margin-left:0px"></p><div><span style=3D"=
font-size:small">We=E2=80=99re hiring if you know of any awesome Java Devop=
s or Linux Operations Engineers!</span><br></div><div><br></div><div>Founde=
r/CEO=C2=A0<a href=3D"http://Spinn3r.com" target=3D"_blank">Spinn3r.com</a>=
<br></div><div>Location:=C2=A0<b>San Francisco, CA</b><br></div><div><font =
color=3D"#2c2c2c" face=3D"Helvetica, Arial, sans-serif"><span style=3D"line=
-height:19px">blog:<b>=C2=A0</b></span></font><a href=3D"http://burtonator.=
wordpress.com" target=3D"_blank">http://burtonator.wordpress.com</a></div><=
div>=E2=80=A6 or check out my <a href=3D"https://plus.google.com/1027182747=
91889610666/posts" target=3D"_blank">Google+ profile</a></div><div><img><br=
></div><div></div><p></p></div></div></div></div></div></div></div></div>
</div></font></span></div>
</blockquote></div><br></div></div></div>
</blockquote></div><br></div>

--001a114e68f026cc630529abbf21--