cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5852) json2sstable breaks on data exported from sstable2json.
Date Wed, 18 Sep 2013 11:38:51 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770679#comment-13770679
] 

Sylvain Lebresne commented on CASSANDRA-5852:
---------------------------------------------

Looking closer at this, I don't think we can change AbstractCompositeType.fromString() like
that. The problem is that ACT.getString() will return an empty string in 2 different cases:
if its argument is one component, but an empty one, and if it's argument is empty. Which means
ACT.fromString() is basically stuck at picking one interpretation over the other and it currently
picks "completely empty" currently.

Now, the problem is that there is cases where we do want ACT.fromString("") to return an empty
byte buffer and that's when you do a select query with CQL2. In that case, an empty byte buffer
is ok (basically empty cell name are not ok, but empty is allowed as the bound of slices)
and probably almost always what you want.

In sstable2json, that's the contrary, you'd want fromString("") to always return a one empty
component buffer, because a stored on disk cell name cannot be empty.

So anyway, just changing ACT.fromString("") will break CQL2 and I'd rather avoid that. One
possible fix would be to change getString() so it generates something different than "" when
given a one empty component name and make fromString understand that. But maybe it's just
not worth the trouble and we should just special case then handling of "" in SSTableImport
directly, before passing it to ACT.fromString.

Other than that, on the patch itself, the corret way to get the type for the value would be
to use: meta.getValueValidator(meta.getColumnDefinitionFromColumnName(...))

                
> json2sstable breaks on data exported from sstable2json.
> -------------------------------------------------------
>
>                 Key: CASSANDRA-5852
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5852
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>    Affects Versions: 1.2.4, 1.2.9, 2.0 rc1
>            Reporter: Ryan McGuire
>            Assignee: Lyuben Todorov
>            Priority: Minor
>             Fix For: 2.0.1
>
>         Attachments: 5852_cassandra-1.2.patch, 5852_cassandra-2.0.patch, r.json
>
>
> Attached is a JSON formatted sstable generated by sstable2json.
> This file cannot be loaded back into Cassandra via json2sstable; it outputs this error:
> {code}
> Counting keys to import, please wait... (NOTE: to skip this use -n <num_keys>)
> Importing 16 keys...
> java.lang.NumberFormatException: Non-hex characters in value6
> 	at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:60)
> 	at org.apache.cassandra.utils.ByteBufferUtil.hexToBytes(ByteBufferUtil.java:503)
> 	at org.apache.cassandra.tools.SSTableImport.stringAsType(SSTableImport.java:578)
> 	at org.apache.cassandra.tools.SSTableImport.access$000(SSTableImport.java:59)
> 	at org.apache.cassandra.tools.SSTableImport$JsonColumn.<init>(SSTableImport.java:154)
> 	at org.apache.cassandra.tools.SSTableImport.addColumnsToCF(SSTableImport.java:231)
> 	at org.apache.cassandra.tools.SSTableImport.addToStandardCF(SSTableImport.java:214)
> 	at org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:432)
> 	at org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:319)
> 	at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:543)
> ERROR: Non-hex characters in value6
> {code}
> Steps to reproduce:
> {code}
> $ ccm create -v git:trunk test-json-import
> Fetching Cassandra updates...
> Current cluster is now: test-json-import
> $ ccm populate -n 1
> $ ccm start
> $ ccm node1 cqlsh
> Connected to test-json-import at 127.0.0.1:9160.
> [cqlsh 4.0.0 | Cassandra 2.0.0-rc1-SNAPSHOT | CQL spec 3.1.0 | Thrift protocol 19.37.0]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor':
1};
> cqlsh> CREATE TABLE test.test (key varchar PRIMARY KEY, value varchar);
> cqlsh> INSERT INTO test.test (key, value) VALUES ('ryan', 'ryan');
> cqlsh> 
> $ ccm node1 flush
> $ ccm stop
> $ ~/.ccm/test-json-import/node1/bin/json2sstable -s -K test -c test ~/Downloads/import_error/r.json
~/.ccm/test-json-import/node1/data/test/test/test-test-ja-1-Data.db 
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message