incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CassUser CassUser <cassu...@gmail.com>
Subject Re: Starting up cassandra 0.7
Date Wed, 17 Nov 2010 01:03:43 GMT
Looking at this closer.  I noticed the following in the SSTableImport Class:

            if (col.isDeleted) {
                cfamily.addColumn(path, hexToBytes(col.value), new
TimestampClock(col.timestamp));
            } else {
                cfamily.addTombstone(path, hexToBytes(col.value), new
TimestampClock(col.timestamp));
            }

This appears to be backwards.


On Tue, Nov 16, 2010 at 4:03 PM, CassUser CassUser <cassuser@gmail.com>wrote:

> Looked at how DatabaseDescriptor is loading the yaml file.  Using that
> approach solves the problem with the column_families mapping exception.
>
> The problems we are running into currently is regarding a known dataset not
> being loaded into our test instance correctly.
>
> Steps:
> 1.  Create temp directory to host cassandra test instance
> 2.  Create keyspace directories
> 3.  Update yaml file, and copy to test location
> 4.  Add schema from yaml using DatabaseDescriptor.
> readTablesFromYaml() & DatabaseDescriptor.
> setTableDefinition
> 4.  Use SSTableImport.importJson to setup a known dataset.  We have JSON
> files converted to 0.7 compliant (byte [] rows)
> 5.  Start Cassandra instance using the EmbeddedCassandraService class
>
> Everything appears to work from the log messages.  I get the message "
> Sampling index for ..", and finally "Listening for thrift clients...".  When
> i use a client to query the data in test instance, I notice keys with no
> columns/values stored (via keyrange slice).  Using a open ended column slice
> for a known key I'm getting zero results.
>
> I'm currently running cassandra 0.7. beta2.  Is the steps I've outlined
> above suppose to work in 0.7?  We did something similar in 0.6.4 without any
> problems.  Is there known bugs I can look into?
>
> Thanks.
>
>
>
> On Tue, Nov 16, 2010 at 12:24 PM, Aaron Morton <aaron@thelastpickle.com>wrote:
>
>> I've not used the embedded service.
>>
>> The code in o.a.c.service.EmbeddedCassandraService says it will read the
>> yaml file. If the cluster does not have a schema stored I think it will load
>> the one from yaml.
>>
>> Have you tried starting it up with an empty system data dir ? Does it
>> pickup the schema from the yaml?
>>
>> Aaron
>>
>> On 17 Nov, 2010,at 09:17 AM, CassUser CassUser <cassuser@gmail.com>
>> wrote:
>>
>> Loading yaml file like so:
>>         FileInputStream yamlInputStream = new FileInputStream(
>>                 configTemplateFile);
>>         Constructor constructor = new Constructor(Config.class);
>>         Yaml yaml = new Yaml(new Loader(constructor));
>>         Config conf = (Config) yaml.load(yamlInputStream);
>>
>> Fails on the last line.
>>
>> Although if i have the CFs defined like this:
>> keyspaces:
>> - column_families:
>>   - !!org.apache.cassandra.config.RawColumnFamily
>>     column_metadata: []
>>     column_type: null
>>     comment: null
>>     compare_subcolumns_with: null
>>     compare_with: BytesType
>>     default_validation_class: null
>>     gc_grace_seconds: 864000
>>     keys_cached: 200000.0
>>     max_compaction_threshold: 32
>>     min_compaction_threshold: 4
>>     name: Similarity
>>     preload_row_cache: false
>>     read_repair_chance: 1.0
>>     rows_cached: 0.0
>>
>>
>> it appears to startup.  Any idea whats going on here?
>>
>>
>> On Tue, Nov 16, 2010 at 11:58 AM, CassUser CassUser <cassuser@gmail.com>wrote:
>>
>>> This is embedded for testing cassandra 0.7 beta2.  using
>>> EmbeddedCassandraService.
>>>
>>> and manually adding schema programmatically using:
>>>         for (KSMetaData table : DatabaseDescriptor.readTablesFromYaml())
>>> {
>>>             for (CFMetaData cfm : table.cfMetaData().values()) {
>>>                 CFMetaData.map(cfm);
>>>             }
>>>             DatabaseDescriptor.setTableDefinition(table,
>>>                     DatabaseDescriptor.getDefsVersion());
>>>         }
>>>
>>> Is this the correct way to start up a test server, with the schema
>>> loaded?
>>>
>>>
>>>
>>>
>>> On Tue, Nov 16, 2010 at 11:41 AM, Aaron Morton <aaron@thelastpickle.com>wrote:
>>>
>>>> AFAIK the ArrayStoreException is similar to a type mismatch. Is it
>>>> possible you have something mixed up in your class path or source code if
>>>> you built from source?
>>>>
>>>> It looks like the column family info was deserialised into a
>>>> o.a.c.config.RawColumnFamily but when that object was added to the
>>>> RawColumnFamily[] array on o.a.c.config.RawKeyspace if was the wrong type.
>>>>
>>>> Have you tried a clean build ? There are some things in the call stack
>>>> which look custom, are you starting from the command line or is this
>>>> embedded for testing?
>>>>
>>>> Hope that helps.
>>>> Aaron
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 17 Nov, 2010,at 08:09 AM, CassUser CassUser <cassuser@gmail.com>
>>>> wrote:
>>>>
>>>>
>>>> Here is the yaml:
>>>>
>>>> # Cassandra YAML generated from previous config
>>>> # Configuration wiki:
>>>> http://wiki.apache.org/cassandra/StorageConfiguration
>>>>  authenticator: org.apache.cassandraauth.AllowAllAuthenticator
>>>>
>>>> auto_bootstrap: false
>>>> binary_memtable_throughput_in_mb: 256
>>>> cluster_name: Test Cluster
>>>> column_index_size_in_kb: 64
>>>> commitlog_rotation_threshold_in_mb: 128
>>>> commitlog_sync: periodic
>>>> commitlog_sync_period_in_ms: 10000
>>>> compaction_thread_priority: 1
>>>> concurrent_reads: 8
>>>> concurrent_writes: 32
>>>> disk_access_mode: auto
>>>> dynamic_snitch: false
>>>> endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch
>>>> request_scheduler: org.apache.cassandra.scheduler.RoundRobinScheduler
>>>> request_scheduler_id: keyspace
>>>> hinted_handoff_enabled: true
>>>> in_memory_compaction_limit_in_mb: 128
>>>> index_interval: 128
>>>> keyspaces:
>>>>     - name: myKeyspace
>>>>        replica_placement_strategy:
>>>> orgapache.cassandra.locator.SimpleStrategy
>>>>
>>>>
>>>>       replication_factor: 1
>>>>       column_families:
>>>>         - name: Standard1
>>>>           rows_cached: 100
>>>>           keys_cached: 1
>>>>           compare_with: UTF8Type
>>>> memtable_flush_after_mins: 60
>>>> memtable_operations_in_millions: 0.3
>>>> memtable_throughput_in_mb: 64
>>>> partitioner: org.apache.cassandra.dht.RandomPartitioner
>>>> phi_convict_threshold: 8
>>>> rpc_keepalive: true
>>>> rpc_port: 9160
>>>> rpc_timeout_in_ms: 10000
>>>> sliced_buffer_size_in_kb: 64
>>>> snapshot_before_compaction: false
>>>> storage_port: 7000
>>>> thrift_framed_transport_size_in_mb: 15
>>>> thrift_max_message_length_in_mb: 16
>>>>
>>>> and here is the exception i'm receiving:
>>>>
>>>> Caused by: Can't construct a java object for tag:yaml.org,2002:org.apache.cassandra.config.Config;
>>>> exception=Cannot create property=keyspaces for Java
>>>> Bean=org.apache.cassandra.config.Config@12e43f1; Cannot create
>>>> property=column_families for
>>>> JavaBean=org.apache.cassandra.config.RawKeyspace@1a8bd74;
>>>> null
>>>>  in "<reader>", line 3, column 1:
>>>>     authenticator: org.apache.cassan ...
>>>>     ^
>>>>
>>>>         at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:372)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177)
>>>>          at
>>>> org.yaml.snakeyamlconstructor.BaseConstructorconstructDocument(BaseConstructor.java:136)
>>>>
>>>>         at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:122)
>>>>         at org.yaml.snakeyaml.Loader.load(Loader.java:52)
>>>>         at org.yaml.snakeyaml.Yaml.load(Yaml.java:166)
>>>>         at
>>>> com.atsid.cassandra.testutils.CassandraTestRunner.updateYamlConfig(CassandraTestRunner.java:134)
>>>>         at
>>>> com.atsid.cassandra.testutils.CassandraTestRunner.init(CassandraTestRunner.java:119)
>>>>         at
>>>> com.atsid.cassandra.testutils.CassandraTestRunner.main(CassandraTestRunner.java:272)
>>>>         ... 6 more
>>>> Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create
>>>> property=keyspaces for JavaBean=org.apache.cassandra.config.Config@12e43f1;
>>>> Cannot cr
>>>> eate property=column_families for
>>>> JavaBean=org.apache.cassandra.config.RawKeyspace@1a8bd74; null
>>>>         at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:370)
>>>>         ... 14 more
>>>> Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create
>>>> property=column_families for
>>>> JavaBean=org.apache.cassandra.config.RawKeyspace@1a8bd74
>>>> ; null
>>>>         at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructSequenceStep2(BaseConstructor.java:239)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructSequence(BaseConstructor.java:233)
>>>>         at
>>>> org.yaml.snakeyaml.constructorConstructor$ConstructSequence.construct(Constructor.java:543)
>>>>
>>>>         at
>>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:298)
>>>>         ... 16 more
>>>> Caused by: java.lang.ArrayStoreException
>>>>         at java.lang.System.arraycopy(Native Method)
>>>>         at java.util.Arrays.copyOf(Arrays.java:2763)
>>>>         at java.util.ArrayList.toArray(ArrayList.java:305)
>>>>         at
>>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:301)
>>>>         ... 23 more
>>>>
>>>>
>>>
>>
>

Mime
View raw message