cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Jirsa (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-9148) Issue when modifying UDT
Date Thu, 16 Apr 2015 05:52:59 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14497593#comment-14497593
] 

Jeff Jirsa edited comment on CASSANDRA-9148 at 4/16/15 5:52 AM:
----------------------------------------------------------------

Thanks [~blerer]. 

Uploaded 9148-v2.txt - also available at https://github.com/jeffjirsa/cassandra/commit/cbfccbe7a3f56db47d560afb0bc8882950d2a5f1

Rebased to trunk as of 20150415

Unit tests updated as suggested (I was originally keeping it minimal and matching the naming
of existing tests, but let's get better tests written now instead of waiting on the 3.0 triage).
New patch includes set/list tests.

No longer causing the double migrations when running this quick test script:

{noformat}
create keyspace test2 WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
1}; 
use test2;

CREATE TYPE test1 (foo text);
CREATE TYPE test2 (foo text, bar text);

CREATE TABLE testtable (id text, map1 frozen<map<text,test1>>, primary key (id));
INSERT INTO testtable (id, map1) VALUES ('foo', {'foo':{foo:'bar'}});
CREATE TABLE testtable2 (id text, map2 frozen<map<text,test2>>, primary key(id));
ALTER TYPE test1 add bar text;
{noformat}

The ALTER TYPE statement now results in a the proper number of announced migrations, rather
than announcing migrations for any table that had any UDT, and that announced migration has
the properly updated comparator for the updated UDT.

{noformat}
INFO  [SharedPool-Worker-2] 2015-04-15 22:25:01,882 MigrationManager.java: 
Update table 'test2/testtable' From org.apache.cassandra.config.CFMetaData@1aee5414[cfId=ee34a6a0-e3f8-11e4-af98-95e9d2c49838,
ksName=test2,
cfName=testtable,
cfType=Standard,
comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,
defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=id,
type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, componentIndex=null, indexName=null,
indexType=null}, ColumnDefinition{name=map1, type=org.apache.cassandra.db.marshal.FrozenType(org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UserType(test2,7465737431,666f6f:org.apache.cassandra.db.marshal.UTF8Type))),

kind=REGULAR, componentIndex=0, indexName=null, indexType=null}],
compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},
compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,
caching={"keys":"ALL", "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false]


To 

org.apache.cassandra.config.CFMetaData@3e779c17[cfId=ee34a6a0-e3f8-11e4-af98-95e9d2c49838,
ksName=test2,
cfName=testtable,
cfType=Standard,
comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),
comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,
defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,
columnMetadata=[ColumnDefinition{name=id, type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null}, ColumnDefinition{name=map1, type=org.apache.cassandra.db.marshal.FrozenType(org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UserType(test2,7465737431,666f6f:org.apache.cassandra.db.marshal.UTF8Type,626172:org.apache.cassandra.db.marshal.UTF8Type))),
kind=REGULAR, componentIndex=0, indexName=null, indexType=null}],compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
"rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false]
{noformat}



was (Author: jjirsa):
Thanks [~blerer]. 

Uploaded 9148-v2.txt - also available at https://github.com/jeffjirsa/cassandra/commit/cbfccbe7a3f56db47d560afb0bc8882950d2a5f1

Rebased to trunk as of 20150415

Unit tests updated as suggested (I was originally keeping it minimal and matching the naming
of existing tests, but let's get better tests written now instead of waiting on the 3.0 triage).
New patch includes set/list tests.

No longer causing the double migrations when running this quick test script:

{noformat}
create keyspace test2 WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
1}; 
use test2;

CREATE TYPE test1 (foo text);
CREATE TYPE test2 (foo text, bar text);

CREATE TABLE testtable (id text, map1 frozen<map<text,test1>>, primary key (id));
INSERT INTO testtable (id, map1) VALUES ('foo', {'foo':{foo:'bar'}});
CREATE TABLE testtable2 (id text, map2 frozen<map<text,test2>>, primary key(id));
ALTER TYPE test1 add bar text;
{noformat}

The ALTER TYPE statement now results in a the proper number of announced migrations, rather
than announcing migrations for any table that had any UDT, and that announced migration has
the properly updated comparator for the updated UDT.

{noformat}
INFO  [SharedPool-Worker-2] 2015-04-15 22:25:01,882 MigrationManager.java: Update table 'test2/testtable'
From org.apache.cassandra.config.CFMetaData@1aee5414[cfId=ee34a6a0-e3f8-11e4-af98-95e9d2c49838,ksName=test2,cfName=testtable,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=id,
type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, componentIndex=null, indexName=null,
indexType=null}, ColumnDefinition{name=map1, type=org.apache.cassandra.db.marshal.FrozenType(org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UserType(test2,7465737431,666f6f:org.apache.cassandra.db.marshal.UTF8Type))),
kind=REGULAR, componentIndex=0, indexName=null, indexType=null}],compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
"rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false]
To org.apache.cassandra.config.CFMetaData@3e779c17[cfId=ee34a6a0-e3f8-11e4-af98-95e9d2c49838,ksName=test2,cfName=testtable,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=id,
type=org.apache.cassandra.db.marshal.UTF8Type, kind=PARTITION_KEY, componentIndex=null, indexName=null,
indexType=null}, ColumnDefinition{name=map1, type=org.apache.cassandra.db.marshal.FrozenType(org.apache.cassandra.db.marshal.MapType(org.apache.cassandra.db.marshal.UTF8Type,org.apache.cassandra.db.marshal.UserType(test2,7465737431,666f6f:org.apache.cassandra.db.marshal.UTF8Type,626172:org.apache.cassandra.db.marshal.UTF8Type))),
kind=REGULAR, componentIndex=0, indexName=null, indexType=null}],compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
"rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=false]
{noformat}


> Issue when modifying UDT
> ------------------------
>
>                 Key: CASSANDRA-9148
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9148
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Oskar Kjellin
>            Assignee: Jeff Jirsa
>             Fix For: 2.1.5
>
>         Attachments: 9148-v2.txt, 9148.txt
>
>
> I'm trying out the user defined types but ran into some issues when adding a column to
an existing type.
> Unfortunately I had to scrap the entire cluster so I cannot access it any more.
> After creating the UDT i adde two tables using it. 1 was just using frozen<type>.
The other was using both frozen<type> frozen map<String, type>.
> Then I realized I needed to add a new field to the user type. Then when I tried to put
to any of the two tables (setting all fields to the UDT in the datastax java driver) I got
this error message that I could not find anywhere else but in the cassandra code:
> com.datastax.driver.core.exceptions.InvalidQueryException: Invalid remaining data after
end of UDT value
> I had to scrap my keyspace in order to be able to use it again. Could not even drop one
of the tables.
> I know that they are frozen so we cannot modify the value of individual fields once they
are written but we must be able to modify the schema right?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message