cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ruslan usifov <ruslan.usi...@gmail.com>
Subject Re: Exception when truncate
Date Thu, 17 May 2012 18:40:49 GMT
Its our test machine with one node in cluster:-)

2012/5/17 Jeremy Hanna <jeremy.hanna1234@gmail.com>:
> when doing a truncate, it has to talk to all of the nodes in the ring to perform the
operation.  by the error, it looks like one of the nodes was unreachable for some reason.
 you might do a nodetool ring in the cli do a 'describe cluster;' and see if your ring is
okay.
>
> So I think the operation is just as fast, it just looks like it times out (20 seconds
or something) when trying to perform the command against all of the nodes in the cluster.
>
> On May 17, 2012, at 9:36 AM, ruslan usifov wrote:
>
>> Maybe, something changes in cassandra 1.0.x for truncate mechanism,
>> because in cassandra 0.8 truncate executes much faster on the same
>> data
>>
>> 2012/5/17 Viktor Jevdokimov <Viktor.Jevdokimov@adform.com>:
>>> Truncate flushes all memtables to free up commit logs, and that on all nodes.
So this takes time. Discussed on this list not so long ago.
>>>
>>> Watch for:
>>> https://issues.apache.org/jira/browse/CASSANDRA-3651
>>> https://issues.apache.org/jira/browse/CASSANDRA-4006
>>>
>>>
>>>
>>> Best regards / Pagarbiai
>>>
>>> Viktor Jevdokimov
>>> Senior Developer
>>>
>>> Email: Viktor.Jevdokimov@adform.com
>>> Phone: +370 5 212 3063
>>> Fax: +370 5 261 0453
>>>
>>> J. Jasinskio 16C,
>>> LT-01112 Vilnius,
>>> Lithuania
>>>
>>>
>>>
>>> Disclaimer: The information contained in this message and attachments is intended
solely for the attention and use of the named addressee and may be confidential. If you are
not the intended recipient, you are reminded that the information remains the property of
the sender. You must not use, disclose, distribute, copy, print or rely on this e-mail. If
you have received this message in error, please contact the sender immediately and irrevocably
delete this message and any copies.> -----Original Message-----
>>>> From: ruslan usifov [mailto:ruslan.usifov@gmail.com]
>>>> Sent: Thursday, May 17, 2012 13:06
>>>> To: user@cassandra.apache.org
>>>> Subject: Re: Exception when truncate
>>>>
>>>> Also i miss understand why on empty CF(no any SStable) truncate heavy
>>>> loads disk??
>>>>
>>>> 2012/5/17 ruslan usifov <ruslan.usifov@gmail.com>:
>>>>> Hello
>>>>>
>>>>> I have follow situation on our test server:
>>>>>
>>>>> from cassandra-cli i try to use
>>>>>
>>>>> truncate purchase_history;
>>>>>
>>>>> 3 times i got:
>>>>>
>>>>> [default@township_6waves] truncate purchase_history; null
>>>>> UnavailableException()
>>>>>        at
>>>>> org.apache.cassandra.thrift.Cassandra$truncate_result.read(Cassandra.j
>>>>> ava:20212)
>>>>>        at
>>>>> org.apache.cassandra.thrift.Cassandra$Client.recv_truncate(Cassandra.j
>>>>> ava:1077)
>>>>>        at
>>>>> org.apache.cassandra.thrift.Cassandra$Client.truncate(Cassandra.java:1
>>>>> 052)
>>>>>        at
>>>>> org.apache.cassandra.cli.CliClient.executeTruncate(CliClient.java:1445
>>>>> )
>>>>>        at
>>>>> org.apache.cassandra.cli.CliClient.executeCLIStatement(CliClient.java:
>>>>> 272)
>>>>>        at
>>>>> org.apache.cassandra.cli.CliMain.processStatementInteractive(CliMain.j
>>>>> ava:220)
>>>>>        at org.apache.cassandra.cli.CliMain.main(CliMain.java:348)
>>>>>
>>>>>
>>>>> So this looks that truncate goes very slow and too long, than
>>>>> rpc_timeout_in_ms: 10000 (this can happens because we have very slow
>>>>> disck on test machine)
>>>>>
>>>>> But in in cassandra system log i see follow exception:
>>>>>
>>>>>
>>>>> ERROR [MutationStage:7022] 2012-05-17 12:19:14,356
>>>>> AbstractCassandraDaemon.java (line 139) Fatal exception in thread
>>>>> Thread[MutationStage:7022,5,main]
>>>>> java.io.IOError: java.io.IOException: unable to mkdirs
>>>>>
>>>> /home/cassandra/1.0.0/data/township_6waves/snapshots/1337242754356-
>>>> pur
>>>>> chase_history
>>>>>        at
>>>>>
>>>> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(Column
>>>> F
>>>>> amilyStore.java:1433)
>>>>>        at
>>>>>
>>>> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.j
>>>>> ava:1462)
>>>>>        at
>>>>>
>>>> org.apache.cassandra.db.ColumnFamilyStore.truncate(ColumnFamilyStore.j
>>>>> ava:1657)
>>>>>        at
>>>>>
>>>> org.apache.cassandra.db.TruncateVerbHandler.doVerb(TruncateVerbHandle
>>>> r
>>>>> .java:50)
>>>>>        at
>>>>>
>>>> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.j
>>>>> ava:59)
>>>>>        at
>>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecu
>>>>> tor.java:886)
>>>>>        at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.
>>>>> java:908)
>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>> Caused by: java.io.IOException: unable to mkdirs
>>>>>
>>>> /home/cassandra/1.0.0/data/township_6waves/snapshots/1337242754356-
>>>> pur
>>>>> chase_history
>>>>>        at
>>>>> org.apache.cassandra.io.util.FileUtils.createDirectory(FileUtils.java:
>>>>> 140)
>>>>>        at
>>>>> org.apache.cassandra.io.util.FileUtils.createDirectory(FileUtils.java:
>>>>> 131)
>>>>>        at
>>>>>
>>>> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(Column
>>>> F
>>>>> amilyStore.java:1409)
>>>>>        ... 7 more
>>>>>
>>>>>
>>>>> Also i see that in snapshort dir already exists
>>>>> 1337242754356-purchase_history directory, so i think that snapshort
>>>>> names that generate cassandra not uniquely.
>>>>>
>>>>> PS: We use cassandra 1.0.10 on Ubuntu 10.0.4-LTS
>

Mime
View raw message