cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Boxenhorn <>
Subject Re: Import/Export of Schema Migrations
Date Mon, 16 May 2011 08:08:00 GMT
What you describe below sounds like what I want to do. I think that the only
additional thing I am requesting is to export the migrations from the dev
cluster (since Cassandra already has a table that saves them - I just want
that information!) so I can import it to the other clusters. This would
ensure that my migrations are exactly right, without being dependent on
error-prone human intervention.

To really get rid of human intervention it would be nice to be able to mark
a certain migration with a version name. Then I could say something like,
"export migrations version1.2.3 to version1.2.4" and I would get the exact
migration path from one version to another.

On Mon, May 16, 2011 at 1:04 AM, aaron morton <>wrote:

> <personal preference>
> Not sure what sort of changes you are making, but this is my approach.
> I've always managed database (my sql, sql server whatever) schema as source
> code (SQL DDL statements, CLI script etc). It makes it a lot easier to cold
> start the system, test changes and see who changed what.
> Once you have your initial schema you can hand roll a CLI script to update
>  / drop existing CF's. For the update column family statement all the
> attributes are delta to the current setting, i.e. you do not need to say
> comparator is ascii again. Other than the indexes, you need to specify all
> the indexes again those not included will be dropped.
> If you want to be able to replay multiple schema changes made during dev
> against other clusters my personal approach would be:
> - create a cli script for every change (using update and delete CF),
> prefixed with 000X so you can see the order.
> - manage the scripts in source control
> - sanity check to see if they can be collapsed
> - replay the changes in order when applying them to a cluster.
> (you will still need to manually delete data from dropped cf's)
> changes to conf/cassandra.yaml can be managed using chef
> </person preference>
> Others will have different ideas....
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> @aaronmorton
> On 14 May 2011, at 00:15, David Boxenhorn wrote:
> Actually, I want a way to propagate *any* changes from development to
> staging to production, but schema changes are the most important.
> Could I use 2221 to propagate schema changes by deleting the schema in the
> target cluster, doing "show schema" in the source cluster, redirecting to a
> file, and running the file as a script in the target cluster?
> Of course, I would have to delete the files of dropped CFs by hand
> (something I *hate* to do, because I'm afraid of making a mistake), but it
> would be a big improvement.
> I am open to any other ideas of how to propagate changes from one cluster
> to another in an efficient non-error-prone fashion. Our development
> environment (i.e. development, staging, production) is pretty standard, so
> I'm sure that I'm not the only one with this problem!
> On Fri, May 13, 2011 at 12:51 PM, aaron morton <>wrote:
>> What sort of schema changes are you making?  can you manage them as a CLI
>> script under source control ?
>> You may also be interested in  CASSANDRA-2221.
>> Cheers
>> Aaron
>>  -----------------
>> Aaron Morton
>> Freelance Cassandra Developer
>> @aaronmorton
>> On 12 May 2011, at 20:45, David Boxenhorn wrote:
>> My use case is like this: I have a development cluster, a staging cluster
>> and a production cluster. When I finish a set of migrations (i.e. changes)
>> on the development cluster, I want to apply them to the staging cluster, and
>> eventually the production cluster. I don't want to do it by hand, because
>> it's a painful and error-prone process. What I would like to do is export
>> the last N migrations from the development cluster as a text file, with
>> exactly the same format as the original text commands, and import them to
>> the staging and production clusters.
>> I think the best place to do this might be the CLI, since you would
>> probably want to view your migrations before exporting them. Something like
>> this:
>> show migrations N;                    Shows the last N migrations.
>> export migrations N <fileName>;       Exports the last N migrations to
>> file fileName.
>> import migrations <fileName>;         Imports migrations from fileName.
>> The import process would apply the migrations one at a time giving you
>> feedback like, "applying migration: update column family...". If a migration
>> fails, the process should give an appropriate message and stop.
>> Is anyone else interested in this? I have created a Jira ticket for it
>> here:

View raw message