ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pranas Baliuka <pra...@orangecap.net>
Subject DataStreamer operation failed
Date Mon, 28 Aug 2017 06:42:01 GMT
Im am prototyping Apache Ignite to store some simple data grid with simple
objects:

Key:
  private final int securityId;
  private final long time;

OHLC: 
  private final int securityId;
  private final long time;
  private final double open;
  private final double high;
  private final double low;
  private final double close;
  private final double marketVWAP;

After inserting 200000 to 300000 such key-value entries getting:


[16:33:32]    __________  ________________ 
[16:33:32]   /  _/ ___/ |/ /  _/_  __/ __/ 
[16:33:32]  _/ // (7 7    // /  / / / _/   
[16:33:32] /___/\___/_/|_/___/ /_/ /___/  
[16:33:32] 
[16:33:32] ver. 2.1.0#20170721-sha1:a6ca5c8a
[16:33:32] 2017 Copyright(C) Apache Software Foundation
[16:33:32] 
[16:33:32] Ignite documentation: http://ignite.apache.org
[16:33:32] 
[16:33:32] Quiet mode.
[16:33:32]   ^-- Logging to file
'/Users/pranas/Apps/apache-ignite-fabric-2.1.0-bin/work/log/ignite-2ac0c6b2.log'
[16:33:32]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false
or "-v" to ignite.{sh|bat}
[16:33:32] 
[16:33:32] OS: Mac OS X 10.12.6 x86_64
[16:33:32] VM information: Java(TM) SE Runtime Environment 1.8.0_40-b27
Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.40-b25
[16:33:32] Configured plugins:
[16:33:32]   ^-- None
[16:33:32] 
[16:33:32] Message queue limit is set to 0 which may lead to potential OOMEs
when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to
message queues growth on sender and receiver sides.
[16:33:32] Security status [authentication=off, tls/ssl=off]
[16:33:32] REST protocols do not start on client node. To start the
protocols on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system
property.
[16:33:33] Refer to this page for more performance suggestions:
https://apacheignite.readme.io/docs/jvm-and-system-tuning
[16:33:33] 
[16:33:33] To start Console Management & Monitoring run
ignitevisorcmd.{sh|bat}
[16:33:33] 
[16:33:33] Ignite node started OK (id=2ac0c6b2)
[16:33:33] Topology snapshot [ver=6, servers=1, clients=1, CPUs=8,
heap=11.0GB]
 >>> Apache Ignite node is up and running.
 >>> Simulator - Real code would process journal events ... TODO.
Processed 0 events so far
Processed 100000 events so far
Processed 200000 events so far
[2017-08-28 16:33:46,592][ERROR][data-streamer-#36%null%][DataStreamerImpl]
DataStreamer operation failed.
class org.apache.ignite.IgniteCheckedException: Failed to finish operation
(too many remaps): 32
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:869)
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:834)
	at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:382)
	at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
	at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
	at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:494)
	at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:473)
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1803)
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:333)
	at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
	at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
	at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
	at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1097)
	at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer
request failed [node=74d746ea-6b57-49be-85eb-3064262ab039]
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1792)
	... 8 more
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to marshal
response error, see node log for details.
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.start(DataStreamProcessor.java:102)
	at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1788)
	at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:938)
	at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1896)
	at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1648)
	at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1076)
	at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:994)
	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:880)
	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:779)
	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:649)
	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:618)
	at org.apache.ignite.Ignition.start(Ignition.java:347)
	at com.algoteq.ignite.IgniteNode.main(IgniteNode.java:37)
[16:33:47] (err) Failed to execute compound future reducer:
GridCompoundFuture [rdc=null, initFlag=1, lsnrCalls=0, done=false,
cancelled=false, err=null, futs=[true]]class
org.apache.ignite.IgniteCheckedException: DataStreamer request failed
[node=74d746ea-6b57-49be-85eb-3064262ab039]
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1792)
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:333)
	at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
	at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
	at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
	at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1097)
	at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to marshal
response error, see node log for details.
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.start(DataStreamProcessor.java:102)
	at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1788)
	at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:938)
	at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1896)
	at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1648)
	at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1076)
	at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:994)
	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:880)
	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:779)
	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:649)
	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:618)
	at org.apache.ignite.Ignition.start(Ignition.java:347)
	at com.algoteq.ignite.IgniteNode.main(IgniteNode.java:37)
Exception in thread "main" java.lang.IllegalStateException: Data streamer
has been closed.
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:404)
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:611)
	at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:673)
	at com.algoteq.ignite.IgniteNode.main(IgniteNode.java:57)
	Suppressed: javax.cache.CacheException: class
org.apache.ignite.IgniteCheckedException: Data streamer has been cancelled:
DataStreamerImpl
[rcvr=org.apache.ignite.internal.processors.datastreamer.DataStreamerCacheUpdaters$Individual@361c294e,
ioPlcRslvr=null, cacheName=OHLCV, bufSize=1024, parallelOps=16, timeout=-1,
autoFlushFreq=0, bufMappings={74d746ea-6b57-49be-85eb-3064262ab039=Buffer
[node=TcpDiscoveryNode [id=74d746ea-6b57-49be-85eb-3064262ab039,
addrs=[0:0:0:0:0:0:0:1, 10.0.1.6, 127.0.0.1],
sockAddrs=[/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, /10.0.1.6:47500],
discPort=47500, order=1, intOrder=1, lastExchangeTime=1503902013125,
loc=false, ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false],
isLocNode=false, idGen=4035,
sem=java.util.concurrent.Semaphore@240d0282[Permits = 16],
batchTopVer=AffinityTopologyVersion [topVer=6, minorTopVer=0], entriesCnt=0,
locFutsSize=0, reqsSize=0]}, cacheObjProc=GridProcessorAdapter [],
cacheObjCtx=org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryContext@7aee5e6c,
cancelled=true, failCntr=235443, activeFuts=GridConcurrentHashSet
[elements=[GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=2117103751], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=94452364], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1767674765], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1717547480],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=337912484], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1267082248], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1013734212], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=426212061],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=274059873], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=225137455], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=270925300], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1421957909],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1654104932], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1851532609], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1853633821], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=621449265],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1609269939], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1379939065], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1173681213], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1161615764],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=170123144], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1110359161], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1546180757], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1716909005],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1969238242], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=366878600], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=777205720], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1835529707],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=65172208], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=212927822], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1428874286], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1106675375],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=98818836], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=164733543], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1904273153], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=2096842550],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=277780535], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=618173717], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1279013285], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1349323953],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1224918404], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1218640657], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=294203776], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=203926563],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1579030959], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=879372185], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=78506224], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1663202868],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=2131033138], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1609486039], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=725960838], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=150964565],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=357653585], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1778993228], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1192766628], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=671948173],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=748975217], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=364480205], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1690481990], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1100636920],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1590983023], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=738695357], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1329164354], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=16145357],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1812681178], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1465263219], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1668305954], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1001100746],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1357371578], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1890858939], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=954382324], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=2136419383],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1263102442], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=567612461], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1906635841], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1988962280],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1688242339], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1099667416]]],
jobPda=org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$DataStreamerPda@4a23c7c8,
depCls=null, fut=DataStreamerFuture [super=GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1853484976]],
publicFut=IgniteFuture [orig=DataStreamerFuture [super=GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1853484976]]],
disconnectErr=null, closed=true, lastFlushTime=1503902027656,
skipStore=false, keepBinary=false, maxRemapCnt=32,
remapSem=java.util.concurrent.Semaphore@2ab9cef8[Permits = 2147483647],
remapOwning=false]
		at
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1312)
		at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.close(DataStreamerImpl.java:1166)
		at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.close(DataStreamerImpl.java:1250)
		at com.algoteq.ignite.IgniteNode.main(IgniteNode.java:63)
	Caused by: class org.apache.ignite.IgniteCheckedException: Data streamer
has been cancelled: DataStreamerImpl
[rcvr=org.apache.ignite.internal.processors.datastreamer.DataStreamerCacheUpdaters$Individual@361c294e,
ioPlcRslvr=null, cacheName=OHLCV, bufSize=1024, parallelOps=16, timeout=-1,
autoFlushFreq=0, bufMappings={74d746ea-6b57-49be-85eb-3064262ab039=Buffer
[node=TcpDiscoveryNode [id=74d746ea-6b57-49be-85eb-3064262ab039,
addrs=[0:0:0:0:0:0:0:1, 10.0.1.6, 127.0.0.1],
sockAddrs=[/0:0:0:0:0:0:0:1:47500, /127.0.0.1:47500, /10.0.1.6:47500],
discPort=47500, order=1, intOrder=1, lastExchangeTime=1503902013125,
loc=false, ver=2.1.0#20170721-sha1:a6ca5c8a, isClient=false],
isLocNode=false, idGen=4035,
sem=java.util.concurrent.Semaphore@240d0282[Permits = 16],
batchTopVer=AffinityTopologyVersion [topVer=6, minorTopVer=0], entriesCnt=0,
locFutsSize=0, reqsSize=0]}, cacheObjProc=GridProcessorAdapter [],
cacheObjCtx=org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryContext@7aee5e6c,
cancelled=true, failCntr=235443, activeFuts=GridConcurrentHashSet
[elements=[GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=2117103751], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=94452364], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1767674765], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1717547480],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=337912484], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1267082248], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1013734212], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=426212061],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=274059873], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=225137455], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=270925300], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1421957909],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1654104932], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1851532609], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1853633821], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=621449265],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1609269939], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1379939065], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1173681213], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1161615764],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=170123144], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1110359161], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1546180757], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1716909005],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1969238242], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=366878600], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=777205720], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1835529707],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=65172208], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=212927822], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1428874286], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1106675375],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=98818836], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=164733543], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1904273153], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=2096842550],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=277780535], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=618173717], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1279013285], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1349323953],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1224918404], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1218640657], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=294203776], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=203926563],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1579030959], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=879372185], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=78506224], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1663202868],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=2131033138], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1609486039], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=725960838], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=150964565],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=357653585], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1778993228], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1192766628], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=671948173],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=748975217], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=364480205], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1690481990], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1100636920],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1590983023], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=738695357], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1329164354], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=16145357],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1812681178], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1465263219], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1668305954], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1001100746],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1357371578], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1890858939], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=954382324], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=2136419383],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1263102442], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=567612461], GridFutureAdapter [ignoreInterrupts=false,
state=INIT, res=null, hash=1906635841], GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1988962280],
GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null,
hash=1688242339], GridFutureAdapter [ignoreInterrupts=false, state=INIT,
res=null, hash=1099667416]]],
jobPda=org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$DataStreamerPda@4a23c7c8,
depCls=null, fut=DataStreamerFuture [super=GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1853484976]],
publicFut=IgniteFuture [orig=DataStreamerFuture [super=GridFutureAdapter
[ignoreInterrupts=false, state=INIT, res=null, hash=1853484976]]],
disconnectErr=null, closed=true, lastFlushTime=1503902027656,
skipStore=false, keepBinary=false, maxRemapCnt=32,
remapSem=java.util.concurrent.Semaphore@2ab9cef8[Permits = 2147483647],
remapOwning=false]
		at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:865)
		at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:834)
		at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:382)
		at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:346)
		at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:334)
		at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:494)
		at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:473)
		at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1803)
		at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:333)
		at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
		at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
		at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
		at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1097)
		at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
		at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
		at java.lang.Thread.run(Thread.java:745)
	Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer
request failed [node=74d746ea-6b57-49be-85eb-3064262ab039]
		at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1792)
		... 8 more
	Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
marshal response error, see node log for details.
		at
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.start(DataStreamProcessor.java:102)
		at
org.apache.ignite.internal.IgniteKernal.startProcessor(IgniteKernal.java:1788)
		at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:938)
		at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1896)
		at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1648)
		at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1076)
		at
org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:994)
		at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:880)
		at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:779)
		at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:649)
		at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:618)
		at org.apache.ignite.Ignition.start(Ignition.java:347)
		at com.algoteq.ignite.IgniteNode.main(IgniteNode.java:37)


My JVM params on 16g RAM:
-XX:+UseG1GC -Xms2g -Xmx9g -XX:MaxDirectMemorySize=512m
-XX:+DisableExplicitGC -XX:+AlwaysPreTouch

config attached ignite-config.xml
<http://apache-ignite-users.70518.x6.nabble.com/file/n16439/ignite-config.xml>  

Ignite 2.1.0:
    compile group: 'org.apache.ignite', name: 'ignite-spark', version:
'2.1.0'

I saw similar issues with very old versions mentioned, but this one is quite
new .... Some new ideas?



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/DataStreamer-operation-failed-tp16439.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Mime
View raw message