incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Data Craftsman <database.crafts...@gmail.com>
Subject Re: Error deleting column families with 1.1
Date Thu, 10 May 2012 04:00:01 GMT
We have similar issue. I'll try to reproduce it and upload logs soon.

On Wed, May 9, 2012 at 3:30 AM, André Cruz <andre.cruz@co.sapo.pt> wrote:
> Here it is: https://issues.apache.org/jira/browse/CASSANDRA-4230
>
> Please let me know if you need further info.
>
> Best regards,
> André
>
> On May 8, 2012, at 23:55 , aaron morton wrote:
>
> Could you please create a ticket for this
> on https://issues.apache.org/jira/browse/CASSANDRA
>
> Please include:
> * operating system
> * keyspace / column family definition
> * output of of "ls -lah" for the "/var/lib/cassandra/data/Disco/Client/"
> directory after the error occurs.
>
> Thanks
>
> -----------------
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
> On 8/05/2012, at 9:00 AM, André Cruz wrote:
>
> Hello.
>
> Since I upgraded to Cassandra 1.1, I get the following error when trying to
> delete a CF. After this happens the CF is not accessible anymore, but I
> cannot create another one with the same name until I restart the server.
>
> INFO [MigrationStage:1] 2012-05-07 18:10:12,682 ColumnFamilyStore.java (line
> 634) Enqueuing flush of Memtable-schema_columnfamilies@1128094887(978/1222
> serialized/live bytes, 21 ops)
> INFO [FlushWriter:2] 2012-05-07 18:10:12,682 Memtable.java (line 266)
> Writing Memtable-schema_columnfamilies@1128094887(978/1222 serialized/live
> bytes, 21 ops)
> INFO [FlushWriter:2] 2012-05-07 18:10:12,720 Memtable.java (line 307)
> Completed flushing
> /var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-28-Data.db
> (1041 bytes)
> INFO [MigrationStage:1] 2012-05-07 18:10:12,721 ColumnFamilyStore.java (line
> 634) Enqueuing flush of Memtable-schema_columns@1599271050(392/490
> serialized/live bytes, 8 ops)
> INFO [FlushWriter:2] 2012-05-07 18:10:12,722 Memtable.java (line 266)
> Writing Memtable-schema_columns@1599271050(392/490 serialized/live bytes, 8
> ops)
> INFO [CompactionExecutor:8] 2012-05-07 18:10:12,722 CompactionTask.java
> (line 114) Compacting
> [SSTableReader(path='/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-26-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-28-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfam
> ilies-hc-27-Data.db'),
> SSTableReader(path='/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-25-Data.db')]
> INFO [FlushWriter:2] 2012-05-07 18:10:12,806 Memtable.java (line 307)
> Completed flushing
> /var/lib/cassandra/data/system/schema_columns/system-schema_columns-hc-23-Data.db
> (447 bytes)
> INFO [CompactionExecutor:8] 2012-05-07 18:10:12,811 CompactionTask.java
> (line 225) Compacted to
> [/var/lib/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-hc-29-Data.db,].
>  24,797 to 21,431
> (~86% of original) bytes for 2 keys at 0.232252MB/s.  Time: 88ms.
> ERROR [MigrationStage:1] 2012-05-07 18:10:12,895 CLibrary.java (line 158)
> Unable to create hard link
> com.sun.jna.LastErrorException: errno was 17
> at org.apache.cassandra.utils.CLibrary.link(Native Method)
> at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:150)
> at
> org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
> at
> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
> at
> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
> at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
> at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
> at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
> at
> org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> ERROR [Thrift:17] 2012-05-07 18:10:12,898 CustomTThreadPoolServer.java (line
> 204) Error occurred during processing of message.
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> java.io.IOError: java.io.IOException: Unable to create hard link from
> /var/lib/cassandra/data/Disco/Client/Client.json to /var/lib/cassandra/data/
> Disco/Client/snapshots/1336410612893-Client/Client.json (errno 17)
> at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:372)
> at
> org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:191)
> at
> org.apache.cassandra.service.MigrationManager.announceColumnFamilyDrop(MigrationManager.java:182)
> at
> org.apache.cassandra.thrift.CassandraServer.system_drop_column_family(CassandraServer.java:948)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3348)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$system_drop_column_family.getResult(Cassandra.java:3336)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOError:
> java.io.IOException: Unable to create hard link from
> /var/lib/cassandra/data/Disco/Client/Client.json to
> /var/lib/cassandra/data/Disco/Client/snapshots/1336410612893-Client/Client.json
> (errno 17)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:368)
> ... 11 more
> Caused by: java.io.IOError: java.io.IOException: Unable to create hard link
> from /var/lib/cassandra/data/Disco/Client/Client.json to
> /var/lib/cassandra/data/Disco/Client/snapshots/1336410612893-Client/Client.json
> (errno 17)
> at
> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
> at
> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
> at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
> at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
> at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
> at
> org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> ... 3 more
> Caused by: java.io.IOException: Unable to create hard link from
> /var/lib/cassandra/data/Disco/Client/Client.json to
> /var/lib/cassandra/data/Disco/Client/snapshots/1336410612893-Client/Client.json
> (errno 17)
> at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
> at
> org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
> at
> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
> ... 10 more
> ERROR [MigrationStage:1] 2012-05-07 18:10:12,899
> AbstractCassandraDaemon.java (line 134) Exception in thread
> Thread[MigrationStage:1,5,main]
> java.io.IOError: java.io.IOException: Unable to create hard link from
> /var/lib/cassandra/data/Disco/Client/Client.json to
> /var/lib/cassandra/data/Disco/Client/snapshots/1336410612893-Client/Client.json
> (errno 17)
> at
> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
> at
> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
> at org.apache.cassandra.db.DefsTable.dropColumnFamily(DefsTable.java:512)
> at org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:403)
> at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:270)
> at
> org.apache.cassandra.service.MigrationManager$1.call(MigrationManager.java:214)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Unable to create hard link from
> /var/lib/cassandra/data/Disco/Client/Client.json to
> /var/lib/cassandra/data/Disco/Client/snapshots/1336410612893-Client/Client.json
> (errno 17)
> at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
> at
> org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
> at
> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
> ... 10 more
>
>
>
> I've tried recreating the data dirs, to see if this was some kind of
> permissions problem, but the error happens every time and with any CF. Has
> anyone seen this error?
>
> I'm using the Cassandra Debian package on Debian squeeze and the Sun JVM
> (build 1.6.0_26-b03).
>
> Best regards,
> André
>
>
>



-- 
--
Thanks,

Charlie (@mujiang) 一个 木匠
=======
Data Architect Developer
http://mujiang.blogspot.com

Mime
View raw message