cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jon Hermes (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (CASSANDRA-1906) Sanitize configuration code
Date Wed, 09 Mar 2011 01:02:12 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004304#comment-13004304
] 

Jon Hermes edited comment on CASSANDRA-1906 at 3/9/11 1:00 AM:
---------------------------------------------------------------

some notes:

CASSANDRA-2007 is the majority of the first part (drop deprecated yaml config). 
Once that's completed, "readTablesFromYaml()" will be gutted, RawKeyspace/ColumnFamily/ColumnDefinition
and schematool removed. YAML is still being used for the node config stuff, but there's much
less magic there (specifically the verbose snakeYAML descriptions of config relations). We
don't need schematool in trunk, but it will have to stay around in 0.7 for 0.6* upgrades.

The tests all use a SchemaLoader to init the keyspaces/cfs before test, and it calls readTablesFromYaml().
This mechanism will need to be changed to completely gut the method. It should instead be
reading from a (static, visible) list of KSMDs passed in by the tests.

--

It doesn't look like Avro is leaving from the migration codepath, so CFMetaData will likely
still have to deal with both thrift and avro objects for now. Builder pattern in CFMetaData
helps gratuitously here to reduce code and confusion, and DefsTest/DD/CFS will all have to
use the new construction pattern. There are two constructors: the standard one requires ks,name,type,and
comparators, and all other values defaulted, private one for system CFs that set specific
cfIDs, then a `CFMD foo( x ) {foo=x;return this}` for each prop foo to set it other than default.

Currently CFMD is mutable (and KSMD is not), so it doesn't specifically need to use a builder
class to produce CFMDs. They're currently mutable because they are held in multiple places
and there's a race condition when migrations come in. This should be simplified (either immutable
and held in one place, or mutable with atomic references), but not for this ticket.


--

For all per-node options:
- Permanent changes are only made by changing the YAML.
- StorageService has all the JMX calls that change DD at runtime.
- To make a permanent change without restarting, change the yaml and make the temp change.
The next time it starts up, it's permanent.
- No thrift involved in node state.

For all per-CF options:
- Permanent changes must be made via migration. Thrift system_* calls, or the cli wrappers
for such.
- Each KSMD sits on its respective Table, each CFMD sits on its respective CFS. SST{R,W} also
needs to be able to access the CFMD in a tight loop. 
Exposing Table (read: Keyspace) changes the MBean hierarchy, and some changes should be made
there as well to make it look more like the actual relations: 
{noformat}
        o.a.c.db (tree root)
       /   |    |      |    \
      SS   CM   HHOM   CL   KEYSPACES
     /   (I/OM)             /   |   \
 Config                    K1   K2  ...
   |   \                  /  \   ...
(D)EPS <NodeOpts>     Table   |
                     /     ColumnFamilies
                <KSOpts>    /     |      \
                          CF1   CF2     ...
                         /  \   ...
                       CFS1  |
                      /    CACHES
                   <CFOpts>   |  \
                             key row 
{noformat}

- Temporary changes are made via JMX (nodetool, Table/CFS beans). They are stronger changes
than new migrations coming in, so a temporary will persist until the node reboots, regardless
of new migrations. To use the settings set in the most recent migration, either reboot the
node or call unsetFoo() (next to getFoo() and setFoo()) for a given property.

      was (Author: jhermes):
    some notes:

CASSANDRA-2007 is the majority of the first part (drop deprecated yaml config). 
Once that's completed, "readTablesFromYaml()" will be gutted, RawKeyspace/ColumnFamily/ColumnDefinition
and schematool removed. YAML is still being used for the node config stuff, but there's much
less magic there (specifically the verbose snakeYAML descriptions of config relations). We
don't need schematool in trunk, but it will have to stay around in 0.7 for 0.6* upgrades.

--

It doesn't look like Avro is leaving from the migration codepath, so CFMetaData will likely
still have to deal with both thrift and avro objects for now. Builder pattern in CFMetaData
helps gratuitously here to reduce code and confusion, and DefsTest/DD/CFS will all have to
use the new construction pattern. There are two constructors: the standard one requires ks,name,type,and
comparators, and all other values defaulted, private one for system CFs that set specific
cfIDs, then a `CFMD foo( x ) {foo=x;return this}` for each prop foo to set it other than default.

Currently CFMD is mutable (and KSMD is not), so it doesn't specifically need to use a builder
class to produce CFMDs. They're currently mutable because they are held in multiple places
and there's a race condition when migrations come in. This should be simplified (either immutable
and held in one place, or mutable with atomic references), but not for this ticket.


--

For all per-node options:
- Permanent changes are only made by changing the YAML.
- StorageService has all the JMX calls that change DD at runtime.
- To make a permanent change without restarting, change the yaml and make the temp change.
The next time it starts up, it's permanent.
- No thrift involved in node state.

For all per-CF options:
- Permanent changes must be made via migration. Thrift system_* calls, or the cli wrappers
for such.
- Each KSMD sits on its respective Table, each CFMD sits on its respective CFS. SST{R,W} also
needs to be able to access the CFMD in a tight loop. 
Exposing Table (read: Keyspace) changes the MBean hierarchy, and some changes should be made
there as well to make it look more like the actual relations: 
{noformat}
        o.a.c.db (tree root)
       /   |    |      |    \
      SS   CM   HHOM   CL   KEYSPACES
     /   (I/OM)             /   |   \
 Config                    K1   K2  ...
   |   \                  /  \   ...
(D)EPS <NodeOpts>     Table   |
                     /     ColumnFamilies
                <KSOpts>    /     |      \
                          CF1   CF2     ...
                         /  \   ...
                       CFS1  |
                      /    CACHES
                   <CFOpts>   |  \
                             key row 
{noformat}

- Temporary changes are made via JMX (nodetool, Table/CFS beans). They are stronger changes
than new migrations coming in, so a temporary will persist until the node reboots, regardless
of new migrations. To use the settings set in the most recent migration, either reboot the
node or call unsetFoo() (next to getFoo() and setFoo()) for a given property.
  
> Sanitize configuration code
> ---------------------------
>
>                 Key: CASSANDRA-1906
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1906
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jon Hermes
>            Assignee: Jon Hermes
>            Priority: Minor
>             Fix For: 0.8
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Multipart:
> - Drop deprecated YAML config. Only config allowed is via thrift/JMX. Make this gratuitously
easy to do with sane defaults and accepting changesets as opposed to full definitions.
> - Combine common code between KS/CF/ColumnDefs and between thrift/avro defs.
> - Provide an obvious and clean interface for changing settings locally versus globally
(JMX vs. thrift). Dox here.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message