cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Strauss <da...@davidstrauss.net>
Subject Re: CassandraFS in 1.0?
Date Thu, 07 Jul 2011 16:37:18 GMT
I'm not sure HDFS has the right properties for a media-storage file
system. We have, however, built a WebDAV server on top of Cassandra
that avoids any pretension of being a general-purpose, POSIX-compliant
file system. We mount it on our servers using davfs2, which is also
nice for a few reasons:

* We can use standard HTTP load-balancing and dead host avoidance
strategies with WebDAV.
* Encrypting access and authenticating clients with PKI/HTTPS works seamlessly.
* WebDAV + davfs2 is etag-header aware, allowing clients to
efficiently validate cached items.
* HTTP is browser and CDN/reverse proxy cache friendly for
distributing content to people who don't need to mount the file
system.
* We could extend the server's support to allow connections from a
broad variety of interactive desktop clients.

On Wed, Jul 6, 2011 at 13:11, Joseph Stein <cryptcom@gmail.com> wrote:
> Hey folks, I am going to start prototyping our media tier using cassandra as
> a file system (meaning upload video/audio/images to web server save in
> cassandra and then streaming them out)
> Has anyone done this before?
> I was thinking brisk's CassandraFS might be a fantastic implementation for
> this but then I feel that I need to run another/different Cassandra cluster
> outside of what our ops folks do with Apache Cassandra 0.8.X
> Am I best to just compress files uploaded to the web server and then start
> chunking and saving chunks in rows and columns so the mem issue does not
> smack me in the face?  And use our existing cluster and build it out
> accordingly?
> I am sure our ops people would like the command line aspect of CassandraFS
> but looking for something that makes the most sense all around.
> It seems to me there is a REALLY great thing in CassandraFS and would love
> to see it as part of 1.0 =8^)  or at a minimum some streamlined
> implementation to-do the same thing.
> If comparing to HDFS that is part of Hadoop project even though Cloudera has
> a distribution of Hadoop :) maybe that can work here too _fingers_crosed_
> (or mongodb->gridfs)
> happy to help as I am moving down this road in general
> Thanks!
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop
> */
>



-- 
David Strauss
   | david@davidstrauss.net
   | +1 512 577 5827 [mobile]

Mime
View raw message