cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Morton <aa...@thelastpickle.com>
Subject Re: Starting up cassandra 0.7
Date Wed, 17 Nov 2010 02:55:59 GMT
It's certainly looks suspect. I've had a look at the code around SSTableImport and SSTableExport
and the isDeleted value for the col is based on IColumn.isMarkedForDelete read when the data
was exported. I'll try to have a look tonight, or if someone is still up in the states they
may help. The current 0.7 branch has the same code. 

Are there any other errors in the log? 

The code sample you've got there uses the TimestampClocks that were once a part of the 0.7
development, but have been removed. So your data sample may not be usable going forwards.
Can you jump to beta3?

Aaron



On 17 Nov, 2010,at 02:03 PM, CassUser CassUser <cassuser@gmail.com> wrote:

Looking at this closer.  I noticed the following in the SSTableImport Class:

            if (col.isDeleted) {
                cfamily.addColumn(path, hexToBytes(col.value), new TimestampClock(col.timestamp));
            } else {
                cfamily.addTombstone(path, hexToBytes(col.value), new TimestampClock(col.timestamp));
            }

This appears to be backwards.


On Tue, Nov 16, 2010 at 4:03 PM, CassUser CassUser <cassuser@gmail.com> wrote:
Looked at how DatabaseDescriptor is loading the yaml file.  Using that approach solves the
problem with the column_families mapping exception.

The problems we are running into currently is regarding a known dataset not being loaded into
our test instance correctly.

Steps:
1.  Create temp directory to host cassandra test instance
2.  Create keyspace directories
3.  Update yaml file, and copy to test location
4.  Add schema from yaml using DatabaseDescriptor.
readTablesFromYaml() & DatabaseDescriptor.
setTableDefinition
4.  Use SSTableImport.importJson to setup a known dataset.  We have JSON files converted
to 0.7 compliant (byte [] rows)
5.  Start Cassandra instance using the EmbeddedCassandraService class

Everything appears to work from the log messages.  I get the message " Sampling index for
..", and finally "Listening for thrift clients...".  When i use a client to query the data
in test instance, I notice keys with no columns/values stored (via keyrange slice).  Using
a open ended column slice for a known key I'm getting zero results.

I'm currently running cassandra 0.7. beta2.  Is the steps I've outlined above suppose to
work in 0.7?  We did something similar in 0.6.4 without any problems.  Is there known bugs
I can look into?

Thanks.




On Tue, Nov 16, 2010 at 12:24 PM, Aaron Morton <aaron@thelastpickle.com> wrote:
I've not used the embedded service. 

The code in o.a.c.service.EmbeddedCassandraService says it will read the yaml file. If the
cluster does not have a schema stored I think it will load the one from yaml. 

Have you tried starting it up with an empty system data dir ? Does it pickup the schema from
the yaml?

Aaron

 
On 17 Nov, 2010,at 09:17 AM, CassUser CassUser <cassuser@gmail.com> wrote:


Loading yaml file like so:
        FileInputStream yamlInputStream = new FileInputStream(
                configTemplateFile);
        Constructor constructor = new Constructor(Config.class);
        Yaml yaml = new Yaml(new Loader(constructor));
        Config conf = (Config) yaml.load(yamlInputStream);

Fails on the last line.

Although if i have the CFs defined like this:
keyspaces:
- column_families:
  - !!org.apache.cassandra.config.RawColumnFamily
    column_metadata: []
    column_type: null
    comment: null
    compare_subcolumns_with: null
    compare_with: BytesType
    default_validation_class: null
    gc_grace_seconds: 864000
    keys_cached: 200000.0
    max_compaction_threshold: 32
    min_compaction_threshold: 4
    name: Similarity
    preload_row_cache: false
    read_repair_chance: 1.0
    rows_cached: 0.0


it appears to startup.  Any idea whats going on here?



On Tue, Nov 16, 2010 at 11:58 AM, CassUser CassUser <cassuser@gmail.com> wrote:

This is embedded for testing cassandra 0.7 beta2.  using EmbeddedCassandraService.

and manually adding schema programmatically using:
        for (KSMetaData table : DatabaseDescriptor.readTablesFromYaml()) {
            for (CFMetaData cfm : table.cfMetaData().values()) {
                CFMetaData.map(cfm);
            }
            DatabaseDescriptor.setTableDefinition(table,
                    DatabaseDescriptor.getDefsVersion());
        }

Is this the correct way to start up a test server, with the schema loaded?





On Tue, Nov 16, 2010 at 11:41 AM, Aaron Morton <aaron@thelastpickle.com> wrote:

AFAIK the ArrayStoreException is similar to a type mismatch. Is it possible you have something
mixed up in your class path or source code if you built from source? 

It looks like the column family info was deserialised into a o.a.c.config.RawColumnFamily
but when that object was added to the RawColumnFamily[] array on o.a.c.config.RawKeyspace
if was the wrong type. 

Have you tried a clean build ? There are some things in the call stack which look custom,
are you starting from the command line or is this embedded for testing?

Hope that helps.
Aaron





On 17 Nov, 2010,at 08:09 AM, CassUser CassUser <cassuser@gmail.com> wrote:



Here is the yaml:

# Cassandra YAML generated from previous config
# Configuration wiki: http://wiki.apache.org/cassandra/StorageConfiguration
authenticator: org.apache.cassandraauth.AllowAllAuthenticator


auto_bootstrap: false
binary_memtable_throughput_in_mb: 256
cluster_name: Test Cluster
column_index_size_in_kb: 64
commitlog_rotation_threshold_in_mb: 128
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
compaction_thread_priority: 1
concurrent_reads: 8
concurrent_writes: 32
disk_access_mode: auto
dynamic_snitch: false
endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch
request_scheduler: org.apache.cassandra.scheduler.RoundRobinScheduler
request_scheduler_id: keyspace
hinted_handoff_enabled: true
in_memory_compaction_limit_in_mb: 128
index_interval: 128
keyspaces:
    - name: myKeyspace

      replica_placement_strategy: orgapache.cassandra.locator.SimpleStrategy


      replication_factor: 1
      column_families:
        - name: Standard1
          rows_cached: 100
          keys_cached: 1
          compare_with: UTF8Type
memtable_flush_after_mins: 60
memtable_operations_in_millions: 0.3
memtable_throughput_in_mb: 64
partitioner: org.apache.cassandra.dht.RandomPartitioner
phi_convict_threshold: 8
rpc_keepalive: true
rpc_port: 9160
rpc_timeout_in_ms: 10000
sliced_buffer_size_in_kb: 64
snapshot_before_compaction: false
storage_port: 7000
thrift_framed_transport_size_in_mb: 15
thrift_max_message_length_in_mb: 16

and here is the exception i'm receiving:

Caused by: Can't construct a java object for tag:yaml.org,2002:org.apache.cassandra.config.Config;
exception=Cannot create property=keyspaces for Java
Bean=org.apache.cassandra.config.Config@12e43f1; Cannot create property=column_families for
JavaBean=org.apache.cassandra.config.RawKeyspace@1a8bd74;
null
 in "<reader>", line 3, column 1:
    authenticator: org.apache.cassan ...
    ^

        at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:372)
        at org.yaml.snakeyaml.constructorBaseConstructor.constructObject(BaseConstructor.java:177)
        at org.yaml.snakeyamlconstructor.BaseConstructorconstructDocument(BaseConstructor.java:136)


        at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:122)
        at org.yaml.snakeyaml.Loader.load(Loader.java:52)
        at org.yaml.snakeyaml.Yaml.load(Yaml.java:166)
        at com.atsid.cassandra.testutils.CassandraTestRunner.updateYamlConfig(CassandraTestRunner.java:134)
        at com.atsid.cassandra.testutils.CassandraTestRunner.init(CassandraTestRunner.java:119)
        at com.atsid.cassandra.testutils.CassandraTestRunner.main(CassandraTestRunner.java:272)
        ... 6 more
Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create property=keyspaces for JavaBean=org.apache.cassandra.config.Config@12e43f1;
Cannot cr
eate property=column_families for JavaBean=org.apache.cassandra.config.RawKeyspace@1a8bd74;
null
        at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305)
        at org.yaml.snakeyamlconstructor.Constructor$ConstructMapping.construct(Constructor.java:184)
        at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:370)
        ... 14 more
Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create property=column_families
for JavaBean=org.apache.cassandra.config.RawKeyspace@1a8bd74
; null
        at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305)
        at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184)
        at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177)
        at org.yaml.snakeyaml.constructor.BaseConstructor.constructSequenceStep2(BaseConstructor.java:239)
        at org.yaml.snakeyaml.constructor.BaseConstructor.constructSequence(BaseConstructor.java:233)
        at org.yaml.snakeyaml.constructorConstructor$ConstructSequence.construct(Constructor.java:543)

        at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177)
        at org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:298)
        ... 16 more
Caused by: java.lang.ArrayStoreException
        at java.lang.System.arraycopy(Native Method)
        at javautil.Arrays.copyOf(Arrays.java:2763)
        at java.utilArrayList.toArray(ArrayList.java:305)
        at org.yamlsnakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:301)
        ... 23 more





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
    • Unnamed multipart/related (inline, None, 0 bytes)
View raw message