cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ali Akhtar <ali.rac...@gmail.com>
Subject Re: Speeding up schema generation during tests
Date Sun, 23 Oct 2016 13:40:08 GMT
I'm using https://github.com/jsevellec/cassandra-unit and haven't come
across any race issues or problems. Cassandra-unit takes care of creating
the schema before it runs the tests.

On Sun, Oct 23, 2016 at 6:17 PM, DuyHai Doan <doanduyhai@gmail.com> wrote:

> Ok I have added -Dcassandra.unsafesystem=true and my tests are broken.
>
> The reason is that I create some schemas before executing tests.
>
> When unable unsafesystem, Cassandra does not block for schema flush so you
> man run into race conditions where the test start using the created schema
> but it has not been fully flushed yet to disk:
>
> See C* source code here: https://github.com/apache/cassandra/blob/trunk/
> src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L278-L282
>
>     static void flush()
>     {
>         if (!DatabaseDescriptor.isUnsafeSystem())
>             ALL.forEach(table -> FBUtilities.waitOnFuture(
> getSchemaCFS(table).forceFlush()));
>     }
>
> I don't know how it worked out for you but it didn't for me...
>
> On Wed, Oct 19, 2016 at 9:45 AM, DuyHai Doan <doanduyhai@gmail.com> wrote:
>
>> Ohh didn't know such system property exist, nice idea!
>>
>> On Wed, Oct 19, 2016 at 9:40 AM, horschi <horschi@gmail.com> wrote:
>>
>>> Have you tried starting Cassandra with -Dcassandra.unsafesystem=true ?
>>>
>>>
>>> On Wed, Oct 19, 2016 at 9:31 AM, DuyHai Doan <doanduyhai@gmail.com>
>>> wrote:
>>>
>>>> As I said, when I bootstrap the server and create some keyspace,
>>>> sometimes the schema is not fully initialized and when the test code tried
>>>> to insert data, it fails.
>>>>
>>>> I did not have time to dig into the source code to find the root cause,
>>>> maybe it's something really stupid and simple to fix. If you want to
>>>> investigate and try out my CassandraDaemon server, I'd be happy to get
>>>> feedbacks
>>>>
>>>> On Wed, Oct 19, 2016 at 9:22 AM, Ali Akhtar <ali.rac200@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks. I've disabled durable writes but this is still pretty slow
>>>>> (about 10 seconds).
>>>>>
>>>>> What issues did you run into with your impl?
>>>>>
>>>>> On Wed, Oct 19, 2016 at 12:15 PM, DuyHai Doan <doanduyhai@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> There is a lot of pre-flight checks when starting the cassandra
>>>>>> server and they took time.
>>>>>>
>>>>>> For integration testing, I have developped a modified CassandraDeamon
>>>>>> here that remove pretty most of those checks:
>>>>>>
>>>>>> https://github.com/doanduyhai/Achilles/blob/master/achilles-
>>>>>> embedded/src/main/java/info/archinnov/achilles/embedded/Achi
>>>>>> llesCassandraDaemon.java
>>>>>>
>>>>>> The problem is that I felt into weird scenarios where creating a
>>>>>> keyspace wasn't created in timely manner so I just stop using this
impl for
>>>>>> the moment, just look at it and do whatever you want.
>>>>>>
>>>>>> Another idea for testing is to disable durable write to speed up
>>>>>> mutation (CREATE KEYSPACE ... WITH durable_write=false)
>>>>>>
>>>>>> On Wed, Oct 19, 2016 at 3:24 AM, Ali Akhtar <ali.rac200@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Is there a way to speed up the creation of keyspace + tables
during
>>>>>>> integration tests? I am using an RF of 1, with SimpleStrategy,
but it still
>>>>>>> takes upto 10-15 seconds.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message