incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <arodr...@gmail.com>
Subject Re: Snapshot failing on JSON files in 1.1.0
Date Wed, 13 Jun 2012 15:28:24 GMT
if I do : locate jna

/opt/java/64/jdk1.6.0_31/db/docs/html/ref/rrefsqljnaturaljoin.html
/root/.m2/repository/net/java/dev/jna
/root/.m2/repository/net/java/dev/jna/jna
/root/.m2/repository/net/java/dev/jna/jna/3.2.7
/root/.m2/repository/net/java/dev/jna/jna/3.2.7/jna-3.2.7-sources.jar
/root/.m2/repository/net/java/dev/jna/jna/3.2.7/jna-3.2.7-sources.jar.sha1
/root/.m2/repository/net/java/dev/jna/jna/3.2.7/jna-3.2.7.jar
/root/.m2/repository/net/java/dev/jna/jna/3.2.7/jna-3.2.7.jar.sha1
/root/.m2/repository/net/java/dev/jna/jna/3.2.7/jna-3.2.7.pom
/root/.m2/repository/net/java/dev/jna/jna/3.2.7/jna-3.2.7.pom.sha1
/usr/share/doc/libjna-java
/usr/share/doc/libjna-java/README.Debian
/usr/share/doc/libjna-java/changelog.Debian.gz
/usr/share/doc/libjna-java/copyright
/usr/share/java/jna-3.2.4.jar
/usr/share/java/jna.jar
/usr/share/maven-repo/net/java/dev/jna
/usr/share/maven-repo/net/java/dev/jna/jna
/usr/share/maven-repo/net/java/dev/jna/jna/3.2.4
/usr/share/maven-repo/net/java/dev/jna/jna/debian
/usr/share/maven-repo/net/java/dev/jna/jna/3.2.4/jna-3.2.4.jar
/usr/share/maven-repo/net/java/dev/jna/jna/3.2.4/jna-3.2.4.pom
/usr/share/maven-repo/net/java/dev/jna/jna/debian/jna-debian.jar
/usr/share/maven-repo/net/java/dev/jna/jna/debian/jna-debian.pom
/var/cache/apt/archives/libjna-java_3.2.4-2_amd64.deb
/var/lib/dpkg/info/libjna-java.list
/var/lib/dpkg/info/libjna-java.md5sums

So what version am I using (jna 3.2.7 or 3.2.4 ?)

Should I do an "apt-get install libjna-java" and does this need a restart ?

Alain

2012/6/13 Alain RODRIGUEZ <arodrime@gmail.com>:
> Hi Aaron,
>
> We are using Ubuntu (AMI Datastax 1.0.9 as I said).
>
> Release:        10.10
> Codename:       maverick
>
> ERROR [RMI TCP Connection(37732)-10.248.10.94] 2012-06-13 15:00:17,157
> CLibrary.java (line 153) Unable to create hard link
> com.sun.jna.LastErrorException: errno was 1
>        at org.apache.cassandra.utils.CLibrary.link(Native Method)
>        at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:145)
>        at org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:857)
>        at org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1412)
>        at org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1462)
>        at org.apache.cassandra.db.Table.snapshot(Table.java:210)
>        at org.apache.cassandra.service.StorageService.takeSnapshot(StorageService.java:1710)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
>        at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
>        at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
>        at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
>        at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
>        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>        at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
>        at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>        at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
>        at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
>        at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>        at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:303)
>        at sun.rmi.transport.Transport$1.run(Transport.java:159)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:662)
>  WARN [RMI TCP Connection(37732)-10.248.10.94] 2012-06-13 15:00:17,158
> CLibrary.java (line 97) Obsolete version of JNA present; unable to
> read errno. Upgrade to JNA 3.2.7 or later
>
> I see that a JNA update seems necessary. Shouldn't this AMI be fully
> worrking "out of the box" ?
>
> I am going to search how to upgrade JNA.
>
> Alain
>
> 2012/5/31 aaron morton <aaron@thelastpickle.com>:
>> CASSANDRA-4230 is a bug in 1.1
>>
>> I am not aware of issues using snapshot on 1.0.9. But errno 0 is a bit odd.
>>
>> On the server side there should be a log message at ERROR level that
>> contains the string "Unable to create hard link" and the error message. What
>> does that say ?
>>
>> Can you also include the OS version.
>>
>> Cheers
>>
>> -----------------
>> Aaron Morton
>> Freelance Developer
>> @aaronmorton
>> http://www.thelastpickle.com
>>
>> On 28/05/2012, at 9:27 PM, Alain RODRIGUEZ wrote:
>>
>> I have the same error with the last Datastax AMI (1.0.9). Is that the same
>> bug ?
>>
>> Requested snapshot for: cassa_teads
>> Exception in thread "main" java.io.IOError: java.io.IOException:
>> Unable to create hard link from
>> /raid0/cassandra/data/cassa_teads/stats_product-hc-233-Index.db to
>> /raid0/cassandra/data/cassa_teads/snapshots/20120528/stats_product-hc-233-Index.db
>> (errno 0)
>>        at
>> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1433)
>>        at
>> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1462)
>>        at org.apache.cassandra.db.Table.snapshot(Table.java:210)
>>        at
>> org.apache.cassandra.service.StorageService.takeSnapshot(StorageService.java:1710)
>>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>        at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
>>        at
>> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
>>        at
>> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
>>        at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
>>        at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
>>        at
>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>>        at
>> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
>>        at
>> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>>        at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
>>        at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>        at java.lang.reflect.Method.invoke(Method.java:597)
>>        at
>> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:303)
>>        at sun.rmi.transport.Transport$1.run(Transport.java:159)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>>        at
>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>>        at
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>>        at
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>        at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>        at java.lang.Thread.run(Thread.java:662)
>> Caused by: java.io.IOException: Unable to create hard link from
>> /raid0/cassandra/data/cassa_teads/stats_product-hc-233-Index.db to
>> /raid0/cassandra/data/cassa_teads/snapshots/20120528/stats_product-hc-233-Index.db
>> (errno 0)
>>        at
>> org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:158)
>>        at
>> org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:857)
>>        at
>> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1412)
>>        ... 32 more
>>
>> Can we do a snapshot manually (like flushing and after copying all the
>> file into the snapshot folder) ?
>>
>> Alain
>>
>> 2012/5/19 Jonathan Ellis <jbellis@gmail.com>:
>>
>> When these bugs are fixed:
>>
>> https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CASSANDRA+AND+fixVersion+%3D+%221.1.1%22+AND+resolution+%3D+Unresolved+ORDER+BY+due+ASC%2C+priority+DESC%2C+created+ASC&mode=hide
>>
>>
>> On Wed, May 16, 2012 at 6:35 PM, Bryan Fernandez <bfernandezsa@gmail.com>
>> wrote:
>>
>> Does anyone know when 1.1.1 will be released?
>>
>>
>> Thanks.
>>
>>
>> On Tue, May 15, 2012 at 5:40 PM, Brandon Williams <driftx@gmail.com> wrote:
>>
>>
>> Probably https://issues.apache.org/jira/browse/CASSANDRA-4230
>>
>>
>> On Tue, May 15, 2012 at 4:08 PM, Bryan Fernandez <bfernandezsa@gmail.com>
>>
>> wrote:
>>
>> Greetings,
>>
>>
>> We recently upgraded from 1.0.8 to 1.1.0. Everything has been running
>>
>> fine
>>
>> with the exception of snapshots. When attempting to snapshot any of the
>>
>> nodes in our six node cluster we are seeing the following error.
>>
>>
>> [root@cassandra-n6 blotter]# /opt/apache-cassandra-1.1.0/bin/nodetool -h
>>
>> 10.20.50.58 snapshot
>>
>> Requested snapshot for: all keyspaces
>>
>> Exception in thread "main" java.io.IOError: java.io.IOException: Unable
>>
>> to
>>
>> create hard link from
>>
>> /var/lib/cassandra/data/blotter/twitter_users/twitter_users.json to
>>
>>
>> /var/lib/cassandra/data/blotter/twitter_users/snapshots/1337115022389/twitter_users.json
>>
>> (errno 17)
>>
>> at
>>
>>
>> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1454)
>>
>> at
>>
>>
>> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1483)
>>
>> at org.apache.cassandra.db.Table.snapshot(Table.java:205)
>>
>> at
>>
>>
>> org.apache.cassandra.service.StorageService.takeSnapshot(StorageService.java:1793)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>> at
>>
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>
>> at
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>>
>> at
>>
>>
>> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
>>
>> at
>>
>>
>> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
>>
>> at
>>
>>
>> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
>>
>> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
>>
>> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
>>
>> at
>>
>>
>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>>
>> at
>>
>> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>>
>> at
>>
>>
>> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
>>
>> at
>>
>>
>> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>>
>> at
>>
>>
>> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
>>
>> at
>>
>>
>> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
>>
>> at
>>
>>
>> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>> at
>>
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>
>> at
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>>
>> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:303)
>>
>> at sun.rmi.transport.Transport$1.run(Transport.java:159)
>>
>> at java.security.AccessController.doPrivileged(Native Method)
>>
>> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>>
>> at
>>
>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>>
>> at
>>
>>
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>>
>> at
>>
>>
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>>
>> at
>>
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>
>> at
>>
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>
>> at java.lang.Thread.run(Thread.java:662)
>>
>> Caused by: java.io.IOException: Unable to create hard link from
>>
>> /var/lib/cassandra/data/blotter/twitter_users/twitter_users.json to
>>
>>
>> /var/lib/cassandra/data/blotter/twitter_users/snapshots/1337115022389/twitter_users.json
>>
>> (errno 17)
>>
>> at org.apache.cassandra.utils.CLibrary.createHardLink(CLibrary.java:163)
>>
>> at
>>
>>
>> org.apache.cassandra.db.Directories.snapshotLeveledManifest(Directories.java:343)
>>
>> at
>>
>>
>> org.apache.cassandra.db.ColumnFamilyStore.snapshotWithoutFlush(ColumnFamilyStore.java:1450)
>>
>> ... 33 more
>>
>>
>>
>> However, an LS shows that both of these JSON files exist on the
>>
>> filesystem
>>
>> (although slightly different sizes).
>>
>>
>> [root@cassandra-n6 blotter]# ls -al
>>
>> /var/lib/cassandra/data/blotter/twitter_users/twitter_users.json
>>
>> -rw-r--r-- 1 root root 38786 May 15 20:51
>>
>> /var/lib/cassandra/data/blotter/twitter_users/twitter_users.json
>>
>>
>> [root@cassandra-n6 blotter]# ls -al
>>
>>
>> /var/lib/cassandra/data/blotter/twitter_users/snapshots/1337115022389/twitter_users.json
>>
>> -rw-r--r-- 1 root root 38778 May 15 20:50
>>
>>
>> /var/lib/cassandra/data/blotter/twitter_users/snapshots/1337115022389/twitter_users.json
>>
>>
>>
>> We are using Leveled Compaction on the twitter_users CF with I assume is
>>
>> creating the JSON files.
>>
>>
>> [root@cassandra-n6 blotter]# ls -al
>>
>> /var/lib/cassandra/data/blotter/twitter_users/*.json
>>
>> -rw-r--r-- 1 root root 38779 May 15 20:51
>>
>> /var/lib/cassandra/data/blotter/twitter_users/twitter_users.json
>>
>> -rw-r--r-- 1 root root 38779 May 15 20:51
>>
>> /var/lib/cassandra/data/blotter/twitter_users/twitter_users-old.json
>>
>> -rw-r--r-- 1 root root  1040 May 15 20:51
>>
>>
>> /var/lib/cassandra/data/blotter/twitter_users/twitter_users.twitter_user_attributes_screenname_idx.json
>>
>> -rw-r--r-- 1 root root  1046 May 15 20:50
>>
>>
>> /var/lib/cassandra/data/blotter/twitter_users/twitter_users.twitter_user_attributes_screenname_idx-old.json
>>
>>
>>
>> The other column families which are not using Leveled Compaction seem to
>>
>> have their snapshots created successfully.
>>
>>
>> Any ideas other than turning off Leveled Compaction?
>>
>>
>>
>> Thanks,
>>
>>
>> Brian
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Jonathan Ellis
>>
>> Project Chair, Apache Cassandra
>>
>> co-founder of DataStax, the source for professional Cassandra support
>>
>> http://www.datastax.com
>>
>>

Mime
View raw message