cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Igor Novgorodov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-13403) nodetool repair breaks SASI index
Date Tue, 04 Apr 2017 08:52:41 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954847#comment-15954847
] 

Igor Novgorodov commented on CASSANDRA-13403:
---------------------------------------------

Here's the cassandra.yaml with comments removed in case it's needed:
{code}
cluster_name: 'MT-Prod'
num_tokens: 256
allocate_tokens_for_keyspace: cservice
hinted_handoff_enabled: true
max_hint_window_in_ms: 10800000 # 3 hours
hinted_handoff_throttle_in_kb: 1024
max_hints_delivery_threads: 2
hints_directory: /var/lib/cassandra/hints
hints_flush_period_in_ms: 10000
max_hints_file_size_in_mb: 128
batchlog_replay_throttle_in_kb: 1024
authenticator: AllowAllAuthenticator
authorizer: AllowAllAuthorizer
role_manager: CassandraRoleManager
roles_validity_in_ms: 2000
permissions_validity_in_ms: 2000
credentials_validity_in_ms: 2000
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
data_file_directories:
    - /var/lib/cassandra/data
commitlog_directory: /var/lib/cassandra/commitlog
cdc_enabled: false
disk_failure_policy: stop
commit_failure_policy: stop
prepared_statements_cache_size_mb:
thrift_prepared_statements_cache_size_mb:
key_cache_size_in_mb: 2048
key_cache_save_period: 3600
row_cache_class_name: org.apache.cassandra.cache.OHCProvider
row_cache_size_in_mb: 2048
row_cache_save_period: 0
counter_cache_size_in_mb: 0
counter_cache_save_period: 7200
saved_caches_directory: /var/lib/cassandra/saved_caches
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
commitlog_segment_size_in_mb: 32
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "10.1.250.70,10.1.250.71"
concurrent_reads: 256
concurrent_writes: 256
concurrent_counter_writes: 32
concurrent_materialized_view_writes: 128
file_cache_size_in_mb: 512
disk_optimization_strategy: ssd
memtable_allocation_type: heap_buffers
index_summary_capacity_in_mb:
index_summary_resize_interval_in_minutes: 60
trickle_fsync: true
trickle_fsync_interval_in_kb: 10240
storage_port: 7000
ssl_storage_port: 7001
listen_address: 10.1.250.70
start_native_transport: true
native_transport_port: 9042
native_transport_max_threads: 256
start_rpc: false
rpc_address: 10.1.250.70
rpc_port: 9160
rpc_keepalive: true
rpc_server_type: sync
thrift_framed_transport_size_in_mb: 15
incremental_backups: false
snapshot_before_compaction: false
auto_snapshot: true
column_index_size_in_kb: 64
column_index_cache_size_in_kb: 2
concurrent_compactors: 8
compaction_throughput_mb_per_sec: 128
sstable_preemptive_open_interval_in_mb: 50
read_request_timeout_in_ms: 60000
range_request_timeout_in_ms: 60000
write_request_timeout_in_ms: 60000
counter_write_request_timeout_in_ms: 5000
cas_contention_timeout_in_ms: 10000
truncate_request_timeout_in_ms: 60000
request_timeout_in_ms: 10000
slow_query_log_timeout_in_ms: 500
cross_node_timeout: true
endpoint_snitch: GossipingPropertyFileSnitch
dynamic_snitch_update_interval_in_ms: 100 
dynamic_snitch_reset_interval_in_ms: 600000
dynamic_snitch_badness_threshold: 0.1
request_scheduler: org.apache.cassandra.scheduler.RoundRobinScheduler
request_scheduler_options:
    throttle_limit: 384
    default_weight: 1
    weights:
      cservice: 2
request_scheduler_id: keyspace
server_encryption_options:
    internode_encryption: none
    keystore: conf/.keystore
    keystore_password: cassandra
    truststore: conf/.truststore
    truststore_password: cassandra
client_encryption_options:
    enabled: false
    optional: false
    keystore: conf/.keystore
    keystore_password: cassandra
internode_compression: none
inter_dc_tcp_nodelay: true
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800
enable_user_defined_functions: true
enable_scripted_user_defined_functions: true
windows_timer_interval: 1
transparent_data_encryption_options:
    enabled: false
    chunk_length_kb: 64
    cipher: AES/CBC/PKCS5Padding
    key_alias: testing:1
    key_provider: 
      - class_name: org.apache.cassandra.security.JKSKeyProvider
        parameters: 
          - keystore: conf/.keystore
            keystore_password: cassandra
            store_type: JCEKS
            key_password: cassandra
tombstone_warn_threshold: 1000
tombstone_failure_threshold: 100000
batch_size_warn_threshold_in_kb: 5
batch_size_fail_threshold_in_kb: 50
unlogged_batch_across_partitions_warn_threshold: 10
compaction_large_partition_warning_threshold_mb: 100
gc_warn_threshold_in_ms: 1000
back_pressure_enabled: false
back_pressure_strategy:
    - class_name: org.apache.cassandra.net.RateBasedBackPressure
      parameters:
        - high_ratio: 0.90
          factor: 5
          flow: FAST
{code}

> nodetool repair breaks SASI index
> ---------------------------------
>
>                 Key: CASSANDRA-13403
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
>             Project: Cassandra
>          Issue Type: Bug
>          Components: sasi
>         Environment: 3.10
>            Reporter: Igor Novgorodov
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
>     recipient text,
>     bulk_id uuid,
>     datetime_final timestamp,
>     datetime_sent timestamp,
>     request_id uuid,
>     status int,
>     PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
>     AND bloom_filter_fp_chance = 0.01
>     AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
>     AND comment = ''
>     AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
>     AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
>     AND crc_check_chance = 1.0
>     AND dclocal_read_repair_chance = 0.1
>     AND default_time_to_live = 0
>     AND gc_grace_seconds = 864000
>     AND max_index_interval = 2048
>     AND memtable_flush_period_in_ms = 0
>     AND min_index_interval = 128
>     AND read_repair_chance = 0.0
>     AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients (bulk_id) USING
'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id = baa94815-e276-4ca4-adda-5b9734e6c4a5;
                                                
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on each node
in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id = baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows after index
query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing the repair
the wrong way?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message