cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Shook <jsh...@gmail.com>
Subject Re: error during snapshot
Date Tue, 27 Apr 2010 18:13:35 GMT
The allocation of memory may have failed depending on the available virtual
memory, whether or not the memory would have been subsequently accessed by
the process.  Some systems do the work of allocating physical pages only
when they are accessed for the first time. I'm not sure if yours is one of
them.

On Tue, Apr 27, 2010 at 10:45 AM, Lee Parker <lee@socialagency.com> wrote:

> Adding a swapfile fixed the error, but it doesn't look as though the
> process is even using the swap file at all.
>
> Lee Parker
> On Tue, Apr 27, 2010 at 9:49 AM, Eric Hauser <ewhauser@gmail.com> wrote:
>
>> Have you read this?
>>
>> http://forums.sun.com/thread.jspa?messageID=9734530
>>
>> <http://forums.sun.com/thread.jspa?messageID=9734530>I don't think EC2
>> instances have any swap.
>>
>>
>>
>> On Tue, Apr 27, 2010 at 10:16 AM, Lee Parker <lee@socialagency.com>wrote:
>>
>>> Can anyone help with this?  It is preventing me from getting backups of
>>> our cluster.
>>>
>>> Lee Parker
>>> On Mon, Apr 26, 2010 at 10:02 PM, Lee Parker <lee@socialagency.com>wrote:
>>>
>>>> I was attempting to get a snapshot on our cassandra nodes.  I get the
>>>> following error every time I run nodetool ... snapshot.
>>>>
>>>> Exception in thread "main" java.io.IOException: Cannot run program "ln":
>>>> java.io.IOException: error=12, Cannot allocate memory
>>>>  at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
>>>> at
>>>> org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:221)
>>>>  at
>>>> org.apache.cassandra.db.ColumnFamilyStore.snapshot(ColumnFamilyStore.java:1060)
>>>> at org.apache.cassandra.db.Table.snapshot(Table.java:256)
>>>>  at
>>>> org.apache.cassandra.service.StorageService.takeAllSnapshot(StorageService.java:1005)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>  at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>  at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
>>>>  at
>>>> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
>>>> at
>>>> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
>>>>  at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
>>>> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
>>>>  at
>>>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
>>>> at
>>>> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
>>>>  at
>>>> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
>>>> at
>>>> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>>>>  at
>>>> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
>>>> at
>>>> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
>>>>  at
>>>> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>  at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>  at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>>>>  at sun.rmi.transport.Transport$1.run(Transport.java:159)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>  at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>>>> at
>>>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>>>>  at
>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>>>> at
>>>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>>>>  at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>  at java.lang.Thread.run(Thread.java:619)
>>>> Caused by: java.io.IOException: java.io.IOException: error=12, Cannot
>>>> allocate memory
>>>> at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
>>>>  at java.lang.ProcessImpl.start(ProcessImpl.java:65)
>>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
>>>>  ... 34 more
>>>>
>>>> The nodes are both Amazon EC2 Large instances with 7.5G RAM (6 allocated
>>>> for Java heap) with two cores and only 70G of data in casssandra.  They have
>>>> plenty of available RAM and HD space.  Has anyone else run into this error?
>>>>
>>>> Lee Parker
>>>>
>>>
>>>
>>
>

Mime
View raw message