incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Picard <picard.dam...@gmail.com>
Subject Re: Storing files in blob into Cassandra
Date Wed, 22 Jun 2011 08:23:59 GMT
>store your images / documents / etc. somewhere and reference them
>in Cassandra.  That's the consensus that's been bandied about on this
>list quite frequently

Thank you for your answers.

I think I have to detail my configuration. On every server of my cluster, I
deploy :
 - a Cassandra node
 - a Tomcat instance
 - the webapp, deployed on Tomcat
 - Apache httpd, in front of Tomcat with mod_jakarta

In front of these, I use a Round-Robin DNS load balancer which balance
request on every httpd.
Every Tomcat instance can access every Cassandra node, allowing them to deal
with every request.
Data are stored with RandomPartitionner, replication factor is 2.

In my case, it would be very easy to store images in Cassandra because these
images will be accessible everywhere in my cluster. If I store images in
FileSystem, I have to replicate them manually (probably with a distributed
filesystem) on every server (quite complicated). This is why I prefer to
store files into Cassandra.

According to Sylvain, the main thing to know is the max size of a file. In
so far as I am on a web purpose, I can define this max file size to 10 Mb
(HTTP POST max size) without disapointing my users.Furthermore, most of
these files will not exceed 2 or 3 Mb. In such case, do you advise me to
store files in Cassandra ?

Thank you.

2011/6/22 Sylvain Lebresne <sylvain@datastax.com>

> Let's be more precise in saying that this all depends on the
> expected size of the documents. If you know that the documents
> will be on the few hundreds kilobytes mark on average and
> no more than a few megabytes (say < 5MB, even though there is
> no magic number), then storing them as blob will work perfectly
> fine (which is not saying storing them externally with metadata in
> Cassandra won't, but using blobs can be simpler in some cases).
>
> I've very successfully stored tons of images as blobs in Cassandra.
> I just knew they couldn't get super big because the system wasn't
> allowing it.
>
> The point with the size being that each time you will get a document,
> Cassandra will have to load it (entirely) in memory to return it.
>
> --
> Sylvain
>
>
> On Wed, Jun 22, 2011 at 9:22 AM, Sasha Dolgy <sdolgy@gmail.com> wrote:
> >
> http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Storing-photos-images-docs-etc-td6078278.html
> >
> > Of significance from that link (which was great until feeling lucky
> > was removed...):
> >
> > Google of terms cassandra large files + feeling lucky
> >
> http://www.google.com/search?q=cassandra+large+files&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a
> >
> > Yields:
> > http://wiki.apache.org/cassandra/FAQ#large_file_and_blob_storage
> >
> >
> > --- store your images / documents / etc. somewhere and reference them
> > in Cassandra.  That's the consensus that's been bandied about on this
> > list quite frequently.  we employ a solution that uses Amazon S3 for
> > storage and Cassandra as the reference to the meta data and location
> > of the files.  works a treat
> >
> >
> > On Wed, Jun 22, 2011 at 9:07 AM, Damien Picard <picard.damien@gmail.com>
> wrote:
> >> Hi,
> >>
> >> I have to store some files (Images, documents, etc.) for my users in a
> >> webapp. I use Cassandra for all of my data and I would like to know if
> this
> >> is a good idea to store these files into blob on a Cassandra CF ?
> >> Is there some contraindications, or special things to know to achieve
> this ?
> >>
> >> Thank you
> >
>



-- 
Damien Picard
Axeiya Services : http://axeiya.com/
gwt-ckeditor : http://code.google.com/p/gwt-ckeditor/
Mon livre sur GWT : http://axeiya.com/index.php/ouvrage-gwt.html

Mime
View raw message