cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Low (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-8515) Hang at startup when no commitlog space
Date Fri, 09 Jan 2015 18:39:36 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Richard Low updated CASSANDRA-8515:
-----------------------------------
    Description: 
If the commit log directory has no free space, Cassandra hangs on startup.

The main thread is waiting:

{code}
"main" prio=9 tid=0x00007fefe400f800 nid=0x1303 waiting on condition [0x000000010b9c1000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007dc8c5fc8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137)
	at org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299)
	at org.apache.cassandra.db.commitlog.CommitLog.<init>(CommitLog.java:73)
	at org.apache.cassandra.db.commitlog.CommitLog.<clinit>(CommitLog.java:53)
	at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360)
	at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
	at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
	at org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699)
	at org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
	at org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390)
	- locked <0x00000007de2f2ce0> (a java.lang.Class for org.apache.cassandra.db.SystemKeyspace)
	at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384)
	at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
{code}

but COMMIT-LOG-ALLOCATOR is RUNNABLE:

{code}
"COMMIT-LOG-ALLOCATOR" prio=9 tid=0x00007fefe5402800 nid=0x7513 in Object.wait() [0x0000000118252000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116)
	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
	at java.lang.Thread.run(Thread.java:745)
{code}

but making no progress.

This behaviour has changed since 1.2 (see CASSANDRA-5737).

  was:
If the commit log directory has no free space, Cassandra hangs on startup.

The main thread is waiting:

{code}
"main" prio=9 tid=0x00007fefe400f800 nid=0x1303 waiting on condition [0x000000010b9c1000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000007dc8c5fc8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
	at org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137)
	at org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299)
	at org.apache.cassandra.db.commitlog.CommitLog.<init>(CommitLog.java:73)
	at org.apache.cassandra.db.commitlog.CommitLog.<clinit>(CommitLog.java:53)
	at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360)
	at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
	at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
	at org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699)
	at org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
	at org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390)
	- locked <0x00000007de2f2ce0> (a java.lang.Class for org.apache.cassandra.db.SystemKeyspace)
	at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384)
	at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
{code}

but COMMIT-LOG-ALLOCATOR is RUNNABLE:

{code}
"COMMIT-LOG-ALLOCATOR" prio=9 tid=0x00007fefe5402800 nid=0x7513 in Object.wait() [0x0000000118252000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116)
	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
	at java.lang.Thread.run(Thread.java:745)
{code}

but making no progress.

This behaviour has change since 1.2 (see CASSANDRA-5737).


> Hang at startup when no commitlog space
> ---------------------------------------
>
>                 Key: CASSANDRA-8515
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8515
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Richard Low
>             Fix For: 2.0.12
>
>
> If the commit log directory has no free space, Cassandra hangs on startup.
> The main thread is waiting:
> {code}
> "main" prio=9 tid=0x00007fefe400f800 nid=0x1303 waiting on condition [0x000000010b9c1000]
>    java.lang.Thread.State: WAITING (parking)
> 	at sun.misc.Unsafe.park(Native Method)
> 	- parking to wait for  <0x00000007dc8c5fc8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> 	at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> 	at org.apache.cassandra.db.commitlog.CommitLogAllocator.fetchSegment(CommitLogAllocator.java:137)
> 	at org.apache.cassandra.db.commitlog.CommitLog.activateNextSegment(CommitLog.java:299)
> 	at org.apache.cassandra.db.commitlog.CommitLog.<init>(CommitLog.java:73)
> 	at org.apache.cassandra.db.commitlog.CommitLog.<clinit>(CommitLog.java:53)
> 	at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:360)
> 	at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:339)
> 	at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:211)
> 	at org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:699)
> 	at org.apache.cassandra.cql3.QueryProcessor.processInternal(QueryProcessor.java:208)
> 	at org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:390)
> 	- locked <0x00000007de2f2ce0> (a java.lang.Class for org.apache.cassandra.db.SystemKeyspace)
> 	at org.apache.cassandra.config.Schema.updateVersion(Schema.java:384)
> 	at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:532)
> 	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:270)
> 	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
> 	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)
> {code}
> but COMMIT-LOG-ALLOCATOR is RUNNABLE:
> {code}
> "COMMIT-LOG-ALLOCATOR" prio=9 tid=0x00007fefe5402800 nid=0x7513 in Object.wait() [0x0000000118252000]
>    java.lang.Thread.State: RUNNABLE
> 	at org.apache.cassandra.db.commitlog.CommitLogAllocator$1.runMayThrow(CommitLogAllocator.java:116)
> 	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}
> but making no progress.
> This behaviour has changed since 1.2 (see CASSANDRA-5737).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message