Mailing-List: contact dev-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of jasedbrown@gmail.com
 designates 209.85.210.47 as permitted sender)
MIME-Version: 1.0
Date: Fri, 25 Jan 2013 09:23:00 -0800
Message-ID: 
 <CAHjqPuJ6quWe2=jfL140ecKaUxzFi_VoWK55XthM9D=wR4aBoA@mail.gmail.com>
Subject: system schema updates
From: Jason Brown <jasedbrown@gmail.com>
To: dev@cassandra.apache.org
Content-Type: multipart/alternative; boundary=047d7b339cebdfc9ff04d4202dde

--047d7b339cebdfc9ff04d4202dde
Content-Type: text/plain; charset=ISO-8859-1

Hi all,

I'm working on a ticket that will require an update to one of the system
tables. Now that we're using system tables with defined schema, starting
with 1.2, the question becomes how to update existing tables. Currently,
most, if not all, of the system tables have their CREATE TABLE statements
hard-coded in the CFMetaData class. It seems to me that adding an update
statment to this class everytime we make a system change will be a bit
awkward, as well as working in the logic of determining which update to
apply when updating c* versions may get hairy.

Hence, I propose a schema migration system where we keep all system schema
changes stored as flat files (perhaps in a new 'conf/migrations'
directory). Each file would have a unique numeric id (monotinically
incrementing) to identify the migration version, thus implying order
between the migrations. The migration version could be stored in the file
itself, but probably easier to store in the file name, preferably as the
first token in the file to allow easier sorting. (This is how I've done it
before.)

We would then have a new system table to tack the current migration status,
system.migration, which would look something like this:

use system;
create table migration (version int);

At c* startup, it would read the system.migration.version value, and
compare against the migration files' ids. If the stored value is lower, the
higher numbered migration files would be applied. This would be a minor
cost at upgrade time (most of which you would have to incur, anyways,
because of the upgrade), but virtually free any other time. It allows us to
have better visibility into system schema change over time, as well.

I'm happy to jump in and start working on this if the community thinks it a
worthwhile addition.

Thanks,

-Jason Brown
Netflix

--047d7b339cebdfc9ff04d4202dde--