aurora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kevin Sweeney (JIRA)" <>
Subject [jira] [Created] (AURORA-112) Introduce formalized schema migrations into the scheduler
Date Fri, 24 Jan 2014 19:42:38 GMT
Kevin Sweeney created AURORA-112:

             Summary: Introduce formalized schema migrations into the scheduler
                 Key: AURORA-112
             Project: Aurora
          Issue Type: Story
          Components: Scheduler
            Reporter: Kevin Sweeney

Right now a change in the schema for persisted Thrift structs is handled by a rather large
"backfill" method that translates from the old schema to the new one. This is a potential
source of datastore corruption bugs.

Some thoughts on the overall design of this "backfill" mechanism. The first is that any structs
we persist should have a SCHEMA_VERSION or some such identifier that gets incremented on subsequent
version changes (we could use also use heuristics here but I think a monotonically increasing
version number is cleaner). The second is that we should define, essentially, a state machine
that takes a V1 struct to a V2 struct (and potentially provides an inverse operation, but
that's not necessary). Basically a piece of code that knows how to make a valid V1 struct
a valid V2 struct. Part of any schema change will be adding this code, and in this way we
will be able to load arbitrarily old persisted structs while keeping the code to do so relatively
clean (each "backfill migration" is a tiny piece of code).

This message was sent by Atlassian JIRA

View raw message