cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sasha Dolgy <sdo...@gmail.com>
Subject Re: Storing files in blob into Cassandra
Date Thu, 23 Jun 2011 05:43:42 GMT
maybe you want to spend a few minutes reading about Haystack over at
facebook to give you some ideas...

https://www.facebook.com/note.php?note_id=76191543919

Not saying what they've done is the right way... just sayin'

On Thu, Jun 23, 2011 at 6:29 AM, AJ <aj@dude.podzone.net> wrote:
>
> I was thinking of doing the same thing.  But, to compensate for the
> bandwidth usage during the read, I was hoping to find a way for the httpd or
> app server to cache the file either in RAM or on disk so subsequent reads
> could just reference the in-mem cache or local hdd.  I have big data
> requirements, so duplicating the storage of file blobs by adding them to the
> hdd would almost double my storage requirements.  So, the hdd cache would
> have to be limited with the LRU removed periodically.
>
> I was thinking about making the key for each file be a relative file path as
> if it were on disk.  This same path could also be used as it's actual
> location on disk in the local disk cache.  Using a path as the key makes it
> flexible in many ways if I ever change my mind and want to store all files
> on disk, or when backing-up or archiving, etc..
>
> But, I'm rusty on my apache http knowledge but I also thought there was an
> apache cache mod that would use both ram and disk depending on the frequency
> of use.  But, I don't know if you can tell it to "cache this blob like it's
> a file".
>
> Just some thoughts.

Mime
View raw message