incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: readable (not hex encoded) column names using sstable2json
Date Fri, 26 Apr 2013 01:32:53 GMT
> First of all thanks for the response. We’re trying to copy existing data into a keyspace
with a different name on the same server. I’m not sure why our operations team wants this.
You can just rename the files. 

> java.lang.NumberFormatException: Non-hex characters in hertz.246944493-2012
>         at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:59)
>         at org.apache.cassandra.utils.ByteBufferUtil.hexToBytes(ByteBufferUtil.java:496)
>         at org.apache.cassandra.tools.SSTableImport.stringAsType(SSTableImport.java:523)
>         at org.apache.cassandra.tools.SSTableImport.access$000(SSTableImport.java:52)
>         at org.apache.cassandra.tools.SSTableImport$JsonColumn.<init>(SSTableImport.java:106)
It's choking on the column name. 
The meta data for the the SSTableImport process has says that the column name is BytesType,
hence the call to ByteBuffer.hexToBytes(). 

Check the schema in the destination system. (But really just poke the Ops team with a stick
until they copy the files, rename them as you need but keep the numbered files together).

Cheers

 
-----------------
Aaron Morton
Freelance Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 24/04/2013, at 8:08 PM, Hans Melgers <Hans.Melgers@anachron.com> wrote:

> First of all thanks for the response. We’re trying to copy existing data into a keyspace
with a different name on the same server. I’m not sure why our operations team wants this.
>  
> We’re also looking into the sstable copy approach you suggested and that could work.
Still I thinks it’s odd the column names are not hex encoded. Below you’ll find some logging
from the json2sstable run showing the exeception. The json files are not manually edited at
all btw.
>  
> This is the CF definition I used as an example. It’s used to model parent child relations
so the column name are foreign keys. I used this CF as example because it’s nice and small.
 
>  
> create column family ACCOUNT_RECEIVERS
>   with column_type = 'Standard'
>   and comparator = 'UTF8Type's
>   and default_validation_class = 'BytesType'
>   and key_validation_class = 'BytesType'
>   and rows_cached = 0.0
>   and row_cache_save_period = 0
>   and row_cache_keys_to_save = 0
>   and keys_cached = 200000.0
>   and key_cache_save_period = 14400
>   and read_repair_chance = 1.0
>   and gc_grace = 864000
>   and min_compaction_threshold = 4
>   and max_compaction_threshold = 32
>   and replicate_on_write = true
>   and row_cache_provider = 'ConcurrentLinkedHashCacheProvider'
>   and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';
>  
> log:
>  
> ---> sFile : /data/dump/ACCOUNT_RECEIVERS-hd-14-Data.db.json ACCOUNT_RECEIVERS-hd-14-Data.db.json
> bas  -  ACCOUNT  -  /data/dump/ACCOUNT_RECEIVERS-hd-14-Data.db.json  - /data/cassandra-data/bas/ACCOUNT_RECEIVERS-hd-14-Data.db
> Counting keys to import, please wait... (NOTE: to skip this use -n <num_keys>)
> Importing 24735 keys...
> java.lang.NumberFormatException: Non-hex characters in hertz.246944493-2012
>         at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:59)
>         at org.apache.cassandra.utils.ByteBufferUtil.hexToBytes(ByteBufferUtil.java:496)
>         at org.apache.cassandra.tools.SSTableImport.stringAsType(SSTableImport.java:523)
>         at org.apache.cassandra.tools.SSTableImport.access$000(SSTableImport.java:52)
>         at org.apache.cassandra.tools.SSTableImport$JsonColumn.<init>(SSTableImport.java:106)
>         at org.apache.cassandra.tools.SSTableImport.addColumnsToCF(SSTableImport.java:191)
>         at org.apache.cassandra.tools.SSTableImport.addToStandardCF(SSTableImport.java:174)
>         at org.apache.cassandra.tools.SSTableImport.importSorted(SSTableImport.java:362)
>         at org.apache.cassandra.tools.SSTableImport.importJson(SSTableImport.java:255)
>         at org.apache.cassandra.tools.SSTableImport.main(SSTableImport.java:479)
> ERROR: Non-hex characters in hertz.246944493-2012
>  
>  
>  
> Van: aaron morton [mailto:aaron@thelastpickle.com] 
> Verzonden: woensdag 24 april 2013 5:37
> Aan: user@cassandra.apache.org
> Onderwerp: Re: readable (not hex encoded) column names using sstable2json
>  
> What the CF definition ?
> What are the errors you are getting?
>  
> We're trying to move data over to another cluster but this prevents us from doing so.

> Is there a reason you are converting the SSTables to JSON ? 
> You could just copy the sstables. 
>  
> Cheers
>  
> -----------------
> Aaron Morton
> Freelance Cassandra Consultant
> New Zealand
>  
> @aaronmorton
> http://www.thelastpickle.com
>  
> On 23/04/2013, at 9:37 PM, Hans Melgers <Hans.Melgers@anachron.com> wrote:
> 
> 
> Hello,
> 
> Using Cassandra 1.0.7 sstable2json on some tables I get readable column
> names. This leads to problems (java.lang.NumberFormatException: Non-hex
> characters in) when importing later.
> 
> We're trying to move data over to another cluster but this prevents us
> from doing so. Could it have to do with using a custom Serializer<T>?
> 
> Here example output:
> 
> D:\Java\apache-cassandra-1.0.7\bin>sstable2json
> d:\var\lib\cassandra\data2\depsi\ACCOUNT_RECEIVERS-hc-1-Data.db
> {
> "23696423656139323162633138666635343135616161336136373337666639623038633
> 9": [["dep.1205050","",1364383456519006]],
> "23696423396338306562366366383365346162383863623266663830663863643930343
> 2": [["dep.1057162","",1364383456664000]],
> [GOES ON here]
> 
> The value "dep.1205050" is literally what we put in there. It's not hex
> encoded.
> 
> Kind regards,
> Hans Melgers
> 
> 
> 


Mime
View raw message