cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roland Gude <>
Subject AW: Cassandra at Amazon AWS
Date Mon, 21 Jan 2013 08:33:48 GMT
On a side note:
If you are going for priam AND you are using LeveledCompaction think carefully whether you
need incremental backups. The s3 upload cost can be very high because Leveled Compaction tends
to create a lot of files and each put request to s3 costs money. We had this setup in relatively
small cluster of 4 nodes where the switch to leveledcompaction increased backup cost by 800
Euro a month.


Von: Roland Gude []
Gesendet: Freitag, 18. Januar 2013 09:23
Betreff: AW: Cassandra at Amazon AWS

Priam is good for backups but it is another complex (but very good) part to a software stack.
A simple solution is to do regular snapshots (via cron)
Compress them and put them into s3
On the s3 you can simply choose how many days the files are kept.

This can be done with a couple of lines of shellscript. And a simple crontab entry

Von: Marcelo Elias Del Valle []
Gesendet: Freitag, 18. Januar 2013 04:53
Betreff: Re: Cassandra at Amazon AWS

Everyone, thanks a lot for the answer, they helped me a lot.

2013/1/17 Andrey Ilinykh <<>>
I'd recommend Priam.


On Thu, Jan 17, 2013 at 5:44 AM, Adam Venturella <<>>
Jared, how do you guys handle data backups for your ephemeral based cluster?

I'm trying to move to ephemeral drives myself, and that was my last sticking point; asking
how others in the community deal with backup in case the VM explodes.

On Wed, Jan 16, 2013 at 1:21 PM, Jared Biel <<>>
We're currently using Cassandra on EC2 at very low scale (a 2 node
cluster on m1.large instances in two regions.) I don't believe that
EBS is recommended for performance reasons. Also, it's proven to be
very unreliable in the past (most of the big/notable AWS outages were
due to EBS issues.) We've moved 99% of our instances off of EBS.

As other have said, if you require more space in the future it's easy
to add more nodes to the cluster. I've found this page
( very useful in determining the amount
of space each instance type has. Note that by default only one
ephemeral drive is attached and you must specify all ephemeral drives
that you want to use at launch time. Also, you can create a RAID 0 of
all local disks to provide maximum speed and space.

On 16 January 2013 20:42, Marcelo Elias Del Valle <<>>
> Hello,
>    I am currently using hadoop + cassandra at amazon AWS. Cassandra runs on
> EC2 and my hadoop process runs at EMR. For cassandra storage, I am using
> local EC2 EBS disks.
>    My system is running fine for my tests, but to me it's not a good setup
> for production. I need my system to perform well for specially for writes on
> cassandra, but the amount of data could grow really big, taking several Tb
> of total storage.
>     My first guess was using S3 as a storage and I saw this can be done by
> using Cloudian package, but I wouldn't like to become dependent on a
> pre-package solution and I found it's kind of expensive for more than 100Tb:
>     I saw some discussion at internet about using EBS or ephemeral disks for
> storage at Amazon too.
>     My question is: does someone on this list have the same problem as me?
> What are you using as solution to Cassandra's storage when running it at
> Amazon AWS?
>     Any thoughts would be highly appreciatted.
> Best regards,
> --
> Marcelo Elias Del Valle
> - @mvallebr

Marcelo Elias Del Valle - @mvallebr

View raw message