cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9303) Match cassandra-loader options in COPY FROM
Date Tue, 29 Dec 2015 16:07:49 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074033#comment-15074033
] 

Stefania commented on CASSANDRA-9303:
-------------------------------------

The hanging tests were caused by the way in which we run cqlsh in ccm, this [pull request|https://github.com/pcmanus/ccm/pull/432]
fixed it.

The remaining failures were caused by the following two things:

* Handling of temporary files is quite different on Windows, the changes to address this are
only in the dtest code, see the second commit of the [pull request|https://github.com/riptano/cassandra-dtest/pull/724].


* The path names should have been normalized, see [this commit|https://github.com/stef1927/cassandra/commit/295219dfbcf24ece9729030cce6e9638899b2842].

I've also changed a few more things, mostly discovered whilst trying to reproduce CASSANDRA-10938
on Windows:

* Reverted to batching by replica to avoid Cassandra processes using too much CPU. Batching
by replica was changed to batching by partition key during the code review of CASSANDRA-9302
because there is a cost in determining the replicas of each record. However, sending batches
with records on different replicas is probably worst then spending a few cycles in Python
determining the correct replicas. It also allows up to use LOGGED batching, see next point.

* Changed batch type from UNLOGGED to LOGGED to avoid a WARN in the Cassandra log files and
for more consistent failed batch status reporting (even though INSERT should be idempotent,
so this can be changed back to UNLOGGED if performance is impacted too much but it shouldn't
since all parititions should be local).

* Fixed a problem with cassandra-stress that only manifested on Windows and on trunk when
using a custom profile. However the Windows stress launch scripts were incorrect from 2.1
onwards.

I worked on the 2.2 patch and merged upwards. I also cherry-picked back to 2.1 with manual
conflict resolution in bin/cqlsh. Even though we don't support Windows for 2.1 I figured it
was best to fix these problems anyway.

> Match cassandra-loader options in COPY FROM
> -------------------------------------------
>
>                 Key: CASSANDRA-9303
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9303
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Tools
>            Reporter: Jonathan Ellis
>            Assignee: Stefania
>            Priority: Critical
>             Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>         Attachments: dtest.out
>
>
> https://github.com/brianmhess/cassandra-loader added a bunch of options to handle real
world requirements, we should match those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message