ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alper Tekinalp <al...@evam.com>
Subject Re: Failing DataStreamer beacuse of minor version change
Date Fri, 21 Apr 2017 12:54:09 GMT
Hi,

So how should I handle that exception? Should I just ignore it? Is
there a way I can know when the data streamer successfully deliver
after getting that error? I just want to know if data
loading/rebalancing finished succesfully.

On Fri, Apr 21, 2017 at 2:10 PM, Evgenii Zhuravlev
<e.zhuravlev.wk@gmail.com> wrote:
> Hi,
>
> Minor topology version changes when you creating or removing caches and at
> late affinity.
>
> Yes, you see this error in log because minor topoly was changed  while
> datastreamer was running, but you didn't lose any data.
>
> According to javadoc:
>
>>Note that there is no guarantee that data will be delivered after this
>> concrete attempt (e.g., it can fail when topology is changing), but it won't
>> be lost anyway.
>
> Evgenii
>
>
> 2017-04-21 13:24 GMT+03:00 Alper Tekinalp <alper@evam.com>:
>>
>> Hi.
>>
>> First of all what is minor topology version and when does it chage?
>>
>> Below we got an error while loading data with data streamer:
>>
>> 18/Apr/2017 13:50:45   INFO   34391455 [exchange-worker-#31%null%]
>> org.apache.ignite.internal.processors.cache.GridCacheProcessor(L:475) -
>> Started cache [name=RECORD_CACHE_XX, mode=PARTITIONED]
>> ..
>> 18/Apr/2017 13:50:49   ERROR  34394884 [DeploymentWorker-0]
>> com.intellica.evam.engine.dynamic.helpers.DeploymentHelper(L:359) - Scenario
>> data could not be preloaded in distributed deployment for scenario [XX].
>> javax.cache.CacheException: class
>> org.apache.ignite.IgniteCheckedException: Failed to finish operation (too
>> many remaps): 32
>>     at
>> org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1465)
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.close(DataStreamerImpl.java:1160)
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.close(DataStreamerImpl.java:1244)
>>     at
>> com.intellica.evam.engine.db.dao.ScenarioRecordDao.preloadScenarioRecords(ScenarioRecordDao.java:134)
>>     at
>> com.intellica.evam.engine.dynamic.helpers.cache.PreloadHelper.preloadScenarioRecordData(PreloadHelper.java:58)
>>     at
>> com.intellica.evam.engine.dynamic.helpers.cache.PreloadHelper.preloadScenarioData(PreloadHelper.java:24)
>>     at
>> com.intellica.evam.engine.dynamic.helpers.DeploymentHelper.distributedDeploy(DeploymentHelper.java:827)
>>     at
>> com.intellica.evam.engine.dynamic.DeploymentWorker.handleDeploymentEvent(DeploymentWorker.java:83)
>>     at
>> com.intellica.evam.engine.dynamic.DeploymentWorker.run(DeploymentWorker.java:62)
>> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
>> finish operation (too many remaps): 32
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:863)
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:828)
>>     at
>> org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:456)
>>     at
>> org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:439)
>>     at
>> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:271)
>>     at
>> org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners(GridFutureAdapter.java:259)
>>     at
>> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:389)
>>     at
>> org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:355)
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1789)
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:335)
>>     at
>> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1215)
>>     at
>> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:843)
>>     at
>> org.apache.ignite.internal.managers.communication.GridIoManager.access$2100(GridIoManager.java:108)
>>     at
>> org.apache.ignite.internal.managers.communication.GridIoManager$6.run(GridIoManager.java:783)
>>     at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>     at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>     at java.lang.Thread.run(Thread.java:745)
>> Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer
>> request failed [node=6fe302a3-f353-4235-af5a-3708bf240750]
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1777)
>>     ... 8 more
>> Caused by: class org.apache.ignite.IgniteCheckedException: DataStreamer
>> will retry data transfer at stable topology [reqTop=AffinityTopologyVersion
>> [topVer=11, minorTopVer=14], topVer=AffinityTopologyVersion [topVer=11,
>> minorTopVer=15], node=remote]
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:339)
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:297)
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:56)
>>     at
>> org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:86)
>>     ... 7 more
>> 18/Apr/2017 13:50:49   INFO   34394950 [exchange-worker-#31%null%]
>> org.apache.ignite.internal.processors.cache.GridCacheProcessor(L:475) -
>> Started cache [name=FEC_XX, mode=PARTITIONED]
>>
>>
>> I guess the reason seems that when one of the server nodes loads data
>> through data streamer to cache RECORD_CACHE_XX, other server creates cache
>> FEC_XX. Bacause cache creation increases the minor topology version.
>>
>> Is my understanding true?
>> If it is what is the reason?
>>
>> --
>> Alper Tekinalp
>>
>> Software Developer
>> Evam Streaming Analytics
>>
>> Atatürk Mah. Turgut Özal Bulv.
>> Gardenya 5 Plaza K:6 Ataşehir
>> 34758 İSTANBUL
>>
>> Tel:  +90 216 455 01 53 Fax: +90 216 455 01 54
>> www.evam.com.tr
>
>



-- 
Alper Tekinalp

Software Developer
Evam Streaming Analytics

Atatürk Mah. Turgut Özal Bulv.
Gardenya 5 Plaza K:6 Ataşehir
34758 İSTANBUL

Tel:  +90 216 455 01 53 Fax: +90 216 455 01 54
www.evam.com.tr

Mime
View raw message