ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ARomantsov (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (IGNITE-9090) When client node make cache.QueryCursorImpl.getAll they have OOM and continue working
Date Thu, 27 Sep 2018 08:59:01 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ARomantsov updated IGNITE-9090:
-------------------------------
    Fix Version/s: 2.8

> When client node make cache.QueryCursorImpl.getAll they have OOM and continue working
> -------------------------------------------------------------------------------------
>
>                 Key: IGNITE-9090
>                 URL: https://issues.apache.org/jira/browse/IGNITE-9090
>             Project: Ignite
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 2.4
>         Environment: 2 server node, 1 client, 1 cache with 15 kk size
>            Reporter: ARomantsov
>            Priority: Critical
>             Fix For: 2.8
>
>
> {code:java}
> [12:21:22,390][SEVERE][query-#69][GridCacheIoManager] Failed to process message [senderId=30cab4ec-1da7-4e9f-a262-bdfa4d466865,
messageType=class o.a.i.i.processors.cache.query.GridCacheQueryResponse]
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>         at java.lang.Long.valueOf(Long.java:840)
>         at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:250)
>         at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:421)
>         at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90)
>         at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
>         at org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:917)
>         at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
>         at org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
>         at java.io.ObjectInputStream.readObject(ObjectInputStream.java:421)
>         at org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:227)
>         at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
>         at org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1777)
>         at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
>         at org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>         at org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310)
>         at org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
>         at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
>         at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.unmarshalCollection0(GridCacheQueryResponse.java:189)
>         at org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.finishUnmarshal(GridCacheQueryResponse.java:162)
>         at org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1530)
>         at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:576)
>         at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:101)
>         at org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1613)
>         at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
>         at org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
>         at org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
>         at org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
>         at org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
>         at org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [12:21:28,573][INFO][ignite-update-notifier-timer][GridUpdateNotifier] Update status
is not available.
> [12:21:23,759][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 22446
milliseconds.
> [12:21:23,758][INFO][grid-timeout-worker-#39][IgniteKernal]
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
>     ^-- Node [id=c1f087b1, uptime=00:01:25.431]
>     ^-- H/N/C [hosts=2, nodes=3, CPUs=32]
>     ^-- CPU [cur=100%, avg=79.09%, GC=8.93%]
>     ^-- PageMemory [pages=0]
>     ^-- Heap [used=216MB, free=8.57%, comm=236MB]
>     ^-- Non heap [used=62MB, free=-1%, comm=65MB]
>     ^-- Outbound messages queue [size=0]
>     ^-- Public thread pool [active=0, idle=0, qSize=0]
>     ^-- System thread pool [active=0, idle=0, qSize=0]
> [12:21:32,609][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 8800
milliseconds.
> [12:21:36,042][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 3383
milliseconds.
> [12:21:41,883][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 5791
milliseconds.
> [12:21:41,884][SEVERE][grid-timeout-worker-#39][GridTimeoutProcessor] Error when executing
timeout callback: CancelableTask [id=92823e5d461-bae41fe7-1702-4a25-8681-916897029985, endTime=1532596959200,
period=60000, cancel=false, task=o.a.i.i.IgniteKernal$4@c0ec5ba]
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [12:21:43,225][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 1291
milliseconds.
> [12:21:44,553][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 1279
milliseconds.
> [12:21:46,979][SEVERE][grid-timeout-worker-#39][GridTimeoutProcessor] Runtime error caught
during grid runnable execution: GridWorker [name=grid-timeout-worker, igniteInstanceName=null,
finished=false, hashCode=1065113566, interrupted=false, runner=grid-timeout-worker-#39]
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [12:21:48,392][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 3789
milliseconds.
> [12:21:49,712][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 1270
milliseconds.
> [12:21:51,007][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 1245
milliseconds.
> [12:21:52,115][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 1058
milliseconds.
> [12:21:53,427][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 1262
milliseconds.
> [12:21:53,427][SEVERE][tcp-client-disco-sock-reader-#3][TcpDiscoverySpi] Runtime error
caught during grid runnable execution: IgniteSpiThread [name=tcp-client-disco-sock-reader-#3]
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [12:21:54,808][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 1331
milliseconds.
> [12:21:55,594][SEVERE][ignite-license-checker-null][GridEntLicenseProcessor] License
grace/burst period - left 57 mins.
> [12:22:18,203][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 1368
milliseconds.
> [12:22:25,228][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 2370
milliseconds.
> [12:22:28,428][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 3150
milliseconds.
> [12:22:32,748][WARNING][jvm-pause-detector-worker][] Possible too long JVM pause: 4270
milliseconds.
> [12:22:37,250][SEVERE][tcp-client-disco-sock-writer-#2][TcpDiscoverySpi] Failed to send
message: null
> java.io.IOException: Failed to get acknowledge for message: TcpDiscoveryClientMetricsUpdateMessage
[super=TcpDiscoveryAbstractMessage [sndNodeId=null, id=cb823e5d461-c1f087b1-678e-4377-a3d1-4bb5dbdb4c98,
verifierNodeId=null, topVer=0, pendingIdx=0, failedNodes=null, isClient=true]]
>         at org.apache.ignite.spi.discovery.tcp.ClientImpl$SocketWriter.body(ClientImpl.java:1359)
>         at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message