cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: failed to delete commitlog, cassandra can't accept writes
Date Thu, 12 Jul 2012 04:52:46 GMT
I don't think it's related to 4337. 

There is an explicit close call just before the deletion attempt. 

Can you create a ticket on https://issues.apache.org/jira/browse/CASSANDRA with all of the
information you've got here (including the full JVM vendor, version, build). Can you also
check if the file it tries to delete exists ? (I assume it does, otherwise it would be a different
error). 

Thanks for digging into this. 

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 11/07/2012, at 9:36 AM, Frank Hsueh wrote:

> oops; I missed log line:
> 
> >>>>
> ERROR [COMMIT-LOG-ALLOCATOR] 2012-07-10 14:19:39,776 AbstractCassandraDaemon.java (line
134) Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Failed to delete C:\var\lib\cassandra\commitlog\CommitLog-948695923996466.log
> 	at org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:176)
> 	at org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:223)
> 	at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> 	at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Failed to delete C:\var\lib\cassandra\commitlog\CommitLog-948695923996466.log
> 	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:54)
> 	at org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:172)
> 	... 4 more
> <<<<
> 
> 
> On Tue, Jul 10, 2012 at 2:35 PM, Frank Hsueh <frank.hsueh@gmail.com> wrote:
> after reading the JIRA, I decided to use Java 6.
> 
> with Casandra 1.1.2 on Java 6 x64 on Win7 sp1 x64 (all latest versions), after a several
minutes of sustained writes, I see:
> 
> from system.log:
> >>>>
> java.io.IOError: java.io.IOException: Failed to delete C:\var\lib\cassandra\commitlog\CommitLog-948695923996466.log
> 	at org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:176)
> 	at org.apache.cassandra.db.commitlog.CommitLogAllocator$4.run(CommitLogAllocator.java:223)
> 	at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> 	at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Failed to delete C:\var\lib\cassandra\commitlog\CommitLog-948695923996466.log
> 	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:54)
> 	at org.apache.cassandra.db.commitlog.CommitLogSegment.discard(CommitLogSegment.java:172)
> 	... 4 more
> <<<<
> 
> anybody seen this before?  is this related to 4337 ?
> 
> 
> 
> 
> On Sat, Jul 7, 2012 at 6:36 PM, Frank Hsueh <frank.hsueh@gmail.com> wrote:
> bug already reported:
> 
> https://issues.apache.org/jira/browse/CASSANDRA-4337
> 
> 
> 
> On Sat, Jul 7, 2012 at 6:26 PM, Frank Hsueh <frank.hsueh@gmail.com> wrote:
> Hi,
> 
> I'm running Casandra 1.1.2 on Java 7 x64 on Win7 sp1 x64 (all latest versions).  If it
matters, I'm using a late version of Astyanax as my client.
> 
> I'm using 4 threads to write a lot of data into a single CF.
> 
> After several minutes of load (~ 30m at last incident), Cassandra stops accepting writes
(client reports an OperationTimeoutException).  I looked at the logs and I see on the Cassandra
server:
> 
> >>>>
> ERROR 18:00:42,807 Exception in thread Thread[COMMIT-LOG-ALLOCATOR,5,main]
> java.io.IOError: java.io.IOException: Rename from \var\lib\cassandra\commitlog\CommitLog-701533048437587.log
to 703272597990002 failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:127)
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.recycle(CommitLogSegment.java:204)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$2.run(CommitLogAllocator.java:166)
>         at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:95)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.lang.Thread.run(Thread.java:722)
> Caused by: java.io.IOException: Rename from \var\lib\cassandra\commitlog\CommitLog-701533048437587.log
to 703272597990002 failed
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(CommitLogSegment.java:105)
>         ... 5 more
> <<<<
> 
> Anybody else seen this before ?
> 
> 
> -- 
> Frank Hsueh | frank.hsueh@gmail.com
> 
> 
> 
> -- 
> Frank Hsueh | frank.hsueh@gmail.com
> 
> 
> 
> -- 
> Frank Hsueh | frank.hsueh@gmail.com
> 
> 
> 
> -- 
> Frank Hsueh | frank.hsueh@gmail.com


Mime
View raw message