cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <>
Subject Re: Import/Export of Schema Migrations
Date Sun, 15 May 2011 22:04:39 GMT
<personal preference>

Not sure what sort of changes you are making, but this is my approach. 

I've always managed database (my sql, sql server whatever) schema as source code (SQL DDL
statements, CLI script etc). It makes it a lot easier to cold start the system, test changes
and see who changed what. 

Once you have your initial schema you can hand roll a CLI script to update  / drop existing
CF's. For the update column family statement all the attributes are delta to the current setting,
i.e. you do not need to say comparator is ascii again. Other than the indexes, you need to
specify all the indexes again those not included will be dropped. 

If you want to be able to replay multiple schema changes made during dev against other clusters
my personal approach would be:

- create a cli script for every change (using update and delete CF), prefixed with 000X so
you can see the order. 
- manage the scripts in source control
- sanity check to see if they can be collapsed 
- replay the changes in order when applying them to a cluster. 
(you will still need to manually delete data from dropped cf's)
changes to conf/cassandra.yaml can be managed using chef 
</person preference>
Others will have different ideas....

Aaron Morton
Freelance Cassandra Developer

On 14 May 2011, at 00:15, David Boxenhorn wrote:

> Actually, I want a way to propagate *any* changes from development to staging to production,
but schema changes are the most important.
> Could I use 2221 to propagate schema changes by deleting the schema in the target cluster,
doing "show schema" in the source cluster, redirecting to a file, and running the file as
a script in the target cluster? 
> Of course, I would have to delete the files of dropped CFs by hand (something I *hate*
to do, because I'm afraid of making a mistake), but it would be a big improvement. 
> I am open to any other ideas of how to propagate changes from one cluster to another
in an efficient non-error-prone fashion. Our development environment (i.e. development, staging,
production) is pretty standard, so I'm sure that I'm not the only one with this problem! 
> On Fri, May 13, 2011 at 12:51 PM, aaron morton <> wrote:
> What sort of schema changes are you making?  can you manage them as a CLI script under
source control ? 
> You may also be interested in  CASSANDRA-2221.
> Cheers
> Aaron
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> On 12 May 2011, at 20:45, David Boxenhorn wrote:
>> My use case is like this: I have a development cluster, a staging cluster and a production
cluster. When I finish a set of migrations (i.e. changes) on the development cluster, I want
to apply them to the staging cluster, and eventually the production cluster. I don't want
to do it by hand, because it's a painful and error-prone process. What I would like to do
is export the last N migrations from the development cluster as a text file, with exactly
the same format as the original text commands, and import them to the staging and production
>> I think the best place to do this might be the CLI, since you would probably want
to view your migrations before exporting them. Something like this:
>> show migrations N;                    Shows the last N migrations.
>> export migrations N <fileName>;       Exports the last N migrations to file
>> import migrations <fileName>;         Imports migrations from fileName.
>> The import process would apply the migrations one at a time giving you feedback like,
"applying migration: update column family...". If a migration fails, the process should give
an appropriate message and stop.
>> Is anyone else interested in this? I have created a Jira ticket for it here:

View raw message