cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mauri Melato (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-4741) Cassandra stops to persist data
Date Mon, 01 Oct 2012 18:47:07 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mauri Melato updated CASSANDRA-4741:
------------------------------------

    Description: 
Hello,
 
 we have a Cassandra instance running on a single node with Windows XP 32bit.
After some time of normal working and some restart, Cassandra seems to stop to persist data
into the keyspace without raising any error.
It's possible to update/insert rows or columns and you can query and see the updated keyspace
but if you stop and start Cassandra, you get a keyspace 'from the past', always from the same
point in the past.

We were not able to understand how to trigger this inconsistency in the first place but once
you get it, you can't get rid of it if not removing the data directories.
Here in attachment you can find a complete set of Cassandra data directories: 
{code}
   cassandra-data/
   |-- commitlog
   |-- data
   `-- saved_caches
{code}
with which it's possible test the inconsistent scenario. 
We tried this on another clean Win Xp 32 machine and we could get the same issue from the
first run.
On a Win XP 64 and the same dir set, Cassandra didn't get into this issue, but from the log
we saw a difference in the disk access mode:
* XP/64: {{DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap" instead of
'standard'}}
* XP/32: {{DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard}}

Not sure this is the key factor, nevertheless in another test bed we experimented the same
issue even on a WinXP 64 node but we couldn't collect the dataset.

Here a short cassandra-cli history to show the issue:
{code}
[default@NEUTRO_Keyspace] connect 192.168.1.33/9160;
Connected to: "NEUTRO Cluster" on 192.168.1.33/9160

[default@unknown] use neutro_keyspace;
Authenticated to keyspace: NEUTRO_Keyspace

[default@NEUTRO_Keyspace] describe;
Keyspace: NEUTRO_Keyspace:
  Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
  Durable Writes: true
    Options: [replication_factor:1]
  Column Families:
    ColumnFamily: TaskId
      Key Validation Class: org.apache.cassandra.db.marshal.AsciiType
      Default column value validator: org.apache.cassandra.db.marshal.UTF8Type
      Columns sorted by: org.apache.cassandra.db.marshal.AsciiType
      GC grace seconds: 864000
      Compaction min/max thresholds: 4/32
      Read repair chance: 1.0
      DC Local Read repair chance: 0.0
      Replicate on write: true
      Caching: KEYS_ONLY
      Bloom Filter FP chance: default
      Built indexes: []
      Column Metadata:
        Column Name: taskId
          Validation Class: org.apache.cassandra.db.marshal.LongType
      Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
      Compression Options:
        sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor

[default@NEUTRO_Keyspace] list TaskId;
Using default limit of 100
Using default column limit of 100
-------------------
RowKey: NE_MASTER_TASK_ID
=> (column=taskId, value=47, timestamp=1349090424494000)

1 Row Returned.
Elapsed time: 93 msec(s).

[default@NEUTRO_Keyspace] set  taskid['NE_MASTER_TASK_ID']['taskId']='55';
Value inserted.
Elapsed time: 31 msec(s).

[default@NEUTRO_Keyspace] list TaskId;
Using default limit of 100
Using default column limit of 100
-------------------
RowKey: NE_MASTER_TASK_ID
=> (column=taskId, value=55, timestamp=1349108488164000)

1 Row Returned.
Elapsed time: 31 msec(s).
{code}

...Stop and Start Cassandra...


{code}
[default@NEUTRO_Keyspace] connect 192.168.1.33/9160;
Connected to: "NEUTRO Cluster" on 192.168.1.33/9160

[default@unknown] use neutro_keyspace;
Authenticated to keyspace: NEUTRO_Keyspace

[default@NEUTRO_Keyspace] list TaskId;
Using default limit of 100
Using default column limit of 100
-------------------
RowKey: NE_MASTER_TASK_ID
=> (column=taskId, value=47, timestamp=1349090424494000)

1 Row Returned.
Elapsed time: 110 msec(s).
{code}

Thanks for your attention


  was:
Hello,
 
 we have a Cassandra instance running on a single node with Windows XP 32bit.
After some time of normal working and some restart, Cassandra seems to stop to persist data
into the keyspace without raising any error.
It's possible to update/insert rows or columns and you can query and see the updated keyspace
but if you stop and start Cassandra, you get a keyspace 'from the past', always from the same
point in the past.

We were not able to understand how to trigger this inconsistency in the first place but once
you get it, you can't get rid of it if not removing the data directories.
Here in attachment you can find a complete set of Cassandra data directories: 
{code}
   cassandra-data/
   |-- commitlog
   |-- data
   `-- saved_caches
{code}
with which it's possible test the inconsistent scenario. 
We tried this on another clean Win Xp 32 machine and we could get the same issue from the
first run.
On a Win XP 64 and the same dir set, Cassandra didn't get into this issue, but from the log
we saw a difference in the disk access mode:
* XP/64: {{DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap" instead of
'standard'}}
* XP/32: {{DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard}}

Not sure this is the key factor, nevertheless in another test bed we experimented the same
issue even on a WinXP 64 node but we couldn't collect the dataset.

Here a short cassandra-cli history to show the issue:
{code}
[default@NEUTRO_Keyspace] connect 192.168.1.33/9160;
Connected to: "NEUTRO Cluster" on 192.168.1.33/9160

[default@unknown] use neutro_keyspace;
Authenticated to keyspace: NEUTRO_Keyspace

[default@NEUTRO_Keyspace] describe;
Keyspace: NEUTRO_Keyspace:
  Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
  Durable Writes: true
    Options: [replication_factor:1]
  Column Families:
    ColumnFamily: TaskId
      Key Validation Class: org.apache.cassandra.db.marshal.AsciiType
      Default column value validator: org.apache.cassandra.db.marshal.UTF8Type
      Columns sorted by: org.apache.cassandra.db.marshal.AsciiType
      GC grace seconds: 864000
      Compaction min/max thresholds: 4/32
      Read repair chance: 1.0
      DC Local Read repair chance: 0.0
      Replicate on write: true
      Caching: KEYS_ONLY
      Bloom Filter FP chance: default
      Built indexes: []
      Column Metadata:
        Column Name: taskId
          Validation Class: org.apache.cassandra.db.marshal.LongType
      Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
      Compression Options:
        sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor

[default@NEUTRO_Keyspace] list TaskId;
Using default limit of 100
Using default column limit of 100
-------------------
RowKey: NE_MASTER_TASK_ID
=> (column=taskId, value=47, timestamp=1349090424494000)

1 Row Returned.
Elapsed time: 93 msec(s).

[default@NEUTRO_Keyspace] set  taskid['NE_MASTER_TASK_ID']['taskId']='55';
Value inserted.
Elapsed time: 31 msec(s).

[default@NEUTRO_Keyspace] list TaskId;
Using default limit of 100
Using default column limit of 100
-------------------
RowKey: NE_MASTER_TASK_ID
=> (column=taskId, value=55, timestamp=1349108488164000)

1 Row Returned.
Elapsed time: 31 msec(s).
{code}

...Stop and Start Cassandra...


{code}
[default@NEUTRO_Keyspace] connect 192.168.1.33/9160;
Connected to: "NEUTRO Cluster" on 192.168.1.33/9160

[default@unknown] use neutro_keyspace;
Authenticated to keyspace: NEUTRO_Keyspace

[default@NEUTRO_Keyspace] list TaskId;
Using default limit of 100
Using default column limit of 100
-------------------
RowKey: NE_MASTER_TASK_ID
=> (column=taskId, value=47, timestamp=1349090424494000)

1 Row Returned.
Elapsed time: 110 msec(s).
{code}


    
> Cassandra stops to persist data
> -------------------------------
>
>                 Key: CASSANDRA-4741
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4741
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: Windows XP 32 bit
> Cassandra 1.1.4 or 1.1.5
>            Reporter: Mauri Melato
>            Priority: Critical
>             Fix For: 1.1.4, 1.1.5
>
>         Attachments: cassandra-data.minimal.zip
>
>
> Hello,
>  
>  we have a Cassandra instance running on a single node with Windows XP 32bit.
> After some time of normal working and some restart, Cassandra seems to stop to persist
data into the keyspace without raising any error.
> It's possible to update/insert rows or columns and you can query and see the updated
keyspace but if you stop and start Cassandra, you get a keyspace 'from the past', always from
the same point in the past.
> We were not able to understand how to trigger this inconsistency in the first place but
once you get it, you can't get rid of it if not removing the data directories.
> Here in attachment you can find a complete set of Cassandra data directories: 
> {code}
>    cassandra-data/
>    |-- commitlog
>    |-- data
>    `-- saved_caches
> {code}
> with which it's possible test the inconsistent scenario. 
> We tried this on another clean Win Xp 32 machine and we could get the same issue from
the first run.
> On a Win XP 64 and the same dir set, Cassandra didn't get into this issue, but from the
log we saw a difference in the disk access mode:
> * XP/64: {{DiskAccessMode 'auto' determined to be mmap, indexAccessMode is mmap" instead
of 'standard'}}
> * XP/32: {{DiskAccessMode 'auto' determined to be standard, indexAccessMode is standard}}
> Not sure this is the key factor, nevertheless in another test bed we experimented the
same issue even on a WinXP 64 node but we couldn't collect the dataset.
> Here a short cassandra-cli history to show the issue:
> {code}
> [default@NEUTRO_Keyspace] connect 192.168.1.33/9160;
> Connected to: "NEUTRO Cluster" on 192.168.1.33/9160
> [default@unknown] use neutro_keyspace;
> Authenticated to keyspace: NEUTRO_Keyspace
> [default@NEUTRO_Keyspace] describe;
> Keyspace: NEUTRO_Keyspace:
>   Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
>   Durable Writes: true
>     Options: [replication_factor:1]
>   Column Families:
>     ColumnFamily: TaskId
>       Key Validation Class: org.apache.cassandra.db.marshal.AsciiType
>       Default column value validator: org.apache.cassandra.db.marshal.UTF8Type
>       Columns sorted by: org.apache.cassandra.db.marshal.AsciiType
>       GC grace seconds: 864000
>       Compaction min/max thresholds: 4/32
>       Read repair chance: 1.0
>       DC Local Read repair chance: 0.0
>       Replicate on write: true
>       Caching: KEYS_ONLY
>       Bloom Filter FP chance: default
>       Built indexes: []
>       Column Metadata:
>         Column Name: taskId
>           Validation Class: org.apache.cassandra.db.marshal.LongType
>       Compaction Strategy: org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
>       Compression Options:
>         sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor
> [default@NEUTRO_Keyspace] list TaskId;
> Using default limit of 100
> Using default column limit of 100
> -------------------
> RowKey: NE_MASTER_TASK_ID
> => (column=taskId, value=47, timestamp=1349090424494000)
> 1 Row Returned.
> Elapsed time: 93 msec(s).
> [default@NEUTRO_Keyspace] set  taskid['NE_MASTER_TASK_ID']['taskId']='55';
> Value inserted.
> Elapsed time: 31 msec(s).
> [default@NEUTRO_Keyspace] list TaskId;
> Using default limit of 100
> Using default column limit of 100
> -------------------
> RowKey: NE_MASTER_TASK_ID
> => (column=taskId, value=55, timestamp=1349108488164000)
> 1 Row Returned.
> Elapsed time: 31 msec(s).
> {code}
> ...Stop and Start Cassandra...
> {code}
> [default@NEUTRO_Keyspace] connect 192.168.1.33/9160;
> Connected to: "NEUTRO Cluster" on 192.168.1.33/9160
> [default@unknown] use neutro_keyspace;
> Authenticated to keyspace: NEUTRO_Keyspace
> [default@NEUTRO_Keyspace] list TaskId;
> Using default limit of 100
> Using default column limit of 100
> -------------------
> RowKey: NE_MASTER_TASK_ID
> => (column=taskId, value=47, timestamp=1349090424494000)
> 1 Row Returned.
> Elapsed time: 110 msec(s).
> {code}
> Thanks for your attention

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message