cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Florent Clairambault (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-4481) Commitlog not replayed after restart - data lost
Date Fri, 17 Aug 2012 00:11:38 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436450#comment-13436450
] 

Florent Clairambault edited comment on CASSANDRA-4481 at 8/17/12 11:11 AM:
---------------------------------------------------------------------------

This bug is marked as resolved, so we're just documenting something that never happened. We're
not scaring anyone here, we're making sure we have all the documentation to prove that I we
were wrong.

So just to make things clear, I didn't make any kind of change or deletion on my keyspaces.
The two keyspaces were created by code (one with pelops and one with hector) once and never
changed. I know I told I did it with cassandra-cli earlier but it turns out that it was entirely
by code.

While doing some tests, I did delete the keyspaces and in that cases it gives an error that
looks like: "Commit logs for non-existing Column Family 1036 were ignored" (I can't find the
exact error in my logs). 

When I deleted the keyspace files, they were recreated by reading the commit logs (this is
step 4 in my previous report). So I think they were in accordance with the schema stored in
cassandra.

--- 

I wanted to actually test it.

The only last versions I could find were 1.0.11, 1.1.2 and 1.1.3. I created a small testscript
and it definitely works with them. But it would be good to test it with 1.1.1 (which I didn't
find) also.

{code}
#!/bin/sh
apt-get remove --purge cassandra -y
rm -Rf /var/log/cassandra /var/lib/cassandra

if [ ! -f cassandra_1.0.11_all.deb ]; then
  wget http://apache.mirrors.multidist.eu/cassandra/debian/pool/main/c/cassandra/cassandra_1.0.11_all.deb
fi

if [ ! -f cassandra_1.1.2_all.deb ]; then
   wget http://apache.mirrors.multidist.eu/cassandra/debian/pool/main/c/cassandra/cassandra_1.1.2_all.deb
fi

if [ ! -f cassandra_1.1.3_all.deb ]; then
  wget http://apache.mirrors.multidist.eu/cassandra/debian/pool/main/c/cassandra/cassandra_1.1.3_all.deb
fi

wait_for_server() {
  while ! echo exit | nc localhost 9160; do sleep 1; done
}

dpkg -i cassandra_1.0.11_all.deb
tail -f /var/log/cassandra/output.log &

wait_for_server;

cassandra-cli -h localhost <<EOF

create keyspace m2mp;
use m2mp;

create column family Registry
  with column_type = 'Standard'
  and comparator = 'AsciiType'
  and default_validation_class = 'AsciiType'
  and key_validation_class = 'AsciiType';

set Registry['/user/florent']['first']='Florent';
set Registry['/user/florent']['country']='France';
set Registry['/version']['1.0.11']='done';
EOF

cassandra-cli -h localhost -k m2mp <<EOF
list Registry;
exit;
EOF

dpkg -i cassandra_1.1.2_all.deb

wait_for_server;

cassandra-cli -h localhost -k m2mp <<EOF
set Registry['/version']['1.1.2']='done';
list Registry;
exit;
EOF

dpkg -i cassandra_1.1.3_all.deb

wait_for_server;

cassandra-cli -h localhost -k m2mp <<EOF
set Registry['/version']['1.1.3']='done';
list Registry;
exit;
EOF

service cassandra restart

wait_for_server;

cassandra-cli -h localhost -k m2mp <<EOF
list Registry;
exit;
EOF
{code}

In the end I do have:
{quote}
-------------------
RowKey: /user/florent
=> (column=country, value=France, timestamp=1345161343036000)
=> (column=first, value=Florent, timestamp=1345161342992000)
-------------------
RowKey: /version
=> (column=1.0.11, value=done, timestamp=1345161343039000)
=> (column=1.1.2, value=done, timestamp=1345161366935000)
=> (column=1.1.3, value=done, timestamp=1345161389760000)
{quote}

So it's ok. But I would be pretty interested to see if we get the same result if we don't
skip any version.
                
      was (Author: superfc):
    This bug is marked as resolved, so we're just documenting something that never happened.
We're not scaring anyone here, we're making sure we have all the documentation to prove that
I we were wrong.

So just to make things clear, I didn't make any kind of change or deletion on my keyspaces.
The two keyspaces were created by code (one with pelops and one with hector) once and never
changed. I know I told I did it with cassandra-cli earlier but it turns out that it was entirely
by code.

While doing some tests, I did delete the keyspaces and in that cases it gives an error that
looks like: "Commit logs for non-existing Column Family 1036 were ignored" (I can't find the
exact error in my logs). 

When I deleted the keyspace files, they were recreated by reading the commit logs (this is
step 4 in my previous report). So I think they were in accordance with the schema stored in
cassandra.

--- 

I wanted to actually test it.

The only last versions I could find were 1.0.11, 1.1.2 and 1.1.3. I created a small testscript
and it definitely works with them. But it would be good to test it with 1.1.1 (which I didn't
have good access to) also.

{code}
#!/bin/sh
apt-get remove --purge cassandra -y
rm -Rf /var/log/cassandra /var/lib/cassandra

if [ ! -f cassandra_1.0.11_all.deb ]; then
  wget http://apache.mirrors.multidist.eu/cassandra/debian/pool/main/c/cassandra/cassandra_1.0.11_all.deb
fi

if [ ! -f cassandra_1.1.2_all.deb ]; then
   wget http://apache.mirrors.multidist.eu/cassandra/debian/pool/main/c/cassandra/cassandra_1.1.2_all.deb
fi

if [ ! -f cassandra_1.1.3_all.deb ]; then
  wget http://apache.mirrors.multidist.eu/cassandra/debian/pool/main/c/cassandra/cassandra_1.1.3_all.deb
fi

wait_for_server() {
  while ! echo exit | nc localhost 9160; do sleep 1; done
}

dpkg -i cassandra_1.0.11_all.deb
tail -f /var/log/cassandra/output.log &

wait_for_server;

cassandra-cli -h localhost <<EOF

create keyspace m2mp;
use m2mp;

create column family Registry
  with column_type = 'Standard'
  and comparator = 'AsciiType'
  and default_validation_class = 'AsciiType'
  and key_validation_class = 'AsciiType';

set Registry['/user/florent']['first']='Florent';
set Registry['/user/florent']['country']='France';
set Registry['/version']['1.0.11']='done';
EOF

cassandra-cli -h localhost -k m2mp <<EOF
list Registry;
exit;
EOF

dpkg -i cassandra_1.1.2_all.deb

wait_for_server;

cassandra-cli -h localhost -k m2mp <<EOF
set Registry['/version']['1.1.2']='done';
list Registry;
exit;
EOF

dpkg -i cassandra_1.1.3_all.deb

wait_for_server;

cassandra-cli -h localhost -k m2mp <<EOF
set Registry['/version']['1.1.3']='done';
list Registry;
exit;
EOF

service cassandra restart

wait_for_server;

cassandra-cli -h localhost -k m2mp <<EOF
list Registry;
exit;
EOF
{code}

In the end I do have:
{quote}
-------------------
RowKey: /user/florent
=> (column=country, value=France, timestamp=1345161343036000)
=> (column=first, value=Florent, timestamp=1345161342992000)
-------------------
RowKey: /version
=> (column=1.0.11, value=done, timestamp=1345161343039000)
=> (column=1.1.2, value=done, timestamp=1345161366935000)
=> (column=1.1.3, value=done, timestamp=1345161389760000)
{quote}

So it's ok. But I would be pretty interested to see if we get the same result if we don't
skip any version.
                  
> Commitlog not replayed after restart - data lost
> ------------------------------------------------
>
>                 Key: CASSANDRA-4481
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4481
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.1.2
>         Environment: Single node cluster on 64Bit CentOS
>            Reporter: Ivo Mei├čner
>            Priority: Critical
>
> When data is written to the commitlog and I restart the machine, all commited data is
lost that has not been flushed to disk. 
> In the startup logs it says that it replays the commitlog successfully, but the data
is not available then. 
> When I open the commitlog file in an editor I can see the added data, but after the restart
it cannot be fetched from cassandra. 
> {code}
>  INFO 09:59:45,362 Replaying /var/myproject/cassandra/commitlog/CommitLog-83203377067.log
>  INFO 09:59:45,476 Finished reading /var/myproject/cassandra/commitlog/CommitLog-83203377067.log
>  INFO 09:59:45,476 Log replay complete, 0 replayed mutations
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message