karaf-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Perumal (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KARAF-4878) Cellar Hazelcast unresponsive when ETH Down
Date Mon, 05 Dec 2016 05:20:58 GMT
Suresh Perumal created KARAF-4878:
-------------------------------------

             Summary: Cellar Hazelcast unresponsive when ETH Down
                 Key: KARAF-4878
                 URL: https://issues.apache.org/jira/browse/KARAF-4878
             Project: Karaf
          Issue Type: Bug
          Components: cellar-hazelcast
    Affects Versions: 4.0.5
         Environment: Redhat Linux 7.2, CentOS 7.2
            Reporter: Suresh Perumal
            Priority: Blocker


Cluster is configured with 2 Nodes. They are up and running.

As part of fail-over scenario simulation. We are trying to test "ETHERNET down scenario" by
running "/etc/sysconfig/network-scripts/ifdown eth0" command on the first node.

During this scenario we are shutting down the first node where the ETH is  down by using monitoring
scripts(in-house scripts). The second node(Among those two nodes) is kept alive.

Second Node's Hazelcast is not accessible for more than 15 minutes. We are getting bellow
exception and no operation related to Hazelcast is working. Applications whichever uses hazelcast
kept frozen.

Invocation                       | 52 - com.hazelcast - 3.5.2 | [10.249.50.80]:5701 [cellar]
[3.5.2] While asking 'is-executing': Invocation{ serviceName='hz:impl:mapService', op=PutOperation{unacknowledged-alarm},
partitionId=165, replicaIndex=0, tryCount=250, tryPauseMillis=500, invokeCount=1, callTimeout=60000,
target=Address[10.249.50.79]:5701, backupsExpected=0, backupsCompleted=0}
java.util.concurrent.TimeoutException: Call Invocation{ serviceName='hz:impl:mapService',
op=com.hazelcast.spi.impl.operationservice.impl.operations.IsStillExecutingOperation{serviceName='hz:impl:mapService',
partitionId=-1, callId=2114, invocationTime=1480511190143, waitTimeout=-1, callTimeout=5000},
partitionId=-1, replicaIndex=0, tryCount=0, tryPauseMillis=0, invokeCount=1, callTimeout=5000,
target=Address[10.249.50.79]:5701, backupsExpected=0, backupsCompleted=0} encountered a timeout
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveApplicationResponse(InvocationFuture.java:366)[52:com.hazelcast:3.5.2]
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.resolveApplicationResponseOrThrowException(InvocationFuture.java:334)[52:com.hazelcast:3.5.2]
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:225)[52:com.hazelcast:3.5.2]
        at com.hazelcast.spi.impl.operationservice.impl.IsStillRunningService.isOperationExecuting(IsStillRunningService.java:85)[52:com.hazelcast:3.5.2]
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.waitForResponse(InvocationFuture.java:275)[52:com.hazelcast:3.5.2]
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:224)[52:com.hazelcast:3.5.2]
        at com.hazelcast.spi.impl.operationservice.impl.InvocationFuture.get(InvocationFuture.java:204)[52:com.hazelcast:3.5.2]
        at com.hazelcast.map.impl.proxy.MapProxySupport.invokeOperation(MapProxySupport.java:456)[52:com.hazelcast:3.5.2]
        at com.hazelcast.map.impl.proxy.MapProxySupport.putInternal(MapProxySupport.java:417)[52:com.hazelcast:3.5.2]
        at com.hazelcast.map.impl.proxy.MapProxyImpl.put(MapProxyImpl.java:97)[52:com.hazelcast:3.5.2]
        at com.hazelcast.map.impl.proxy.MapProxyImpl.put(MapProxyImpl.java:87)[52:com.hazelcast:3.5.2]
        at com.fujitsu.fnc.emf.fpmplatform.cachemanager.HazelcastCacheManagerMapServiceImpl.addToMap(HazelcastCacheManagerMapServiceImpl.java:87)[209:FPMHazelcastCache:4.1.0.SNAPSHOT]
        at Proxy1897a82c_c032_4a5c_9839_e71cb2af452a.addToMap(Unknown Source)[:]
        at com.fujitsu.fnc.ngemf.fm.server.impl.FpmConsumerTask.prepareJSON(FpmConsumerTask.java:151)[235:com.fujitsu.fnc.ngemf.fm.server.impl:4.1.0.SNAPSHOT]
        at com.fujitsu.fnc.ngemf.fm.server.impl.FpmConsumerTask.run(FpmConsumerTask.java:244)[235:com.fujitsu.fnc.ngemf.fm.server.impl:4.1.0.SNAPSHOT]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_66]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_66]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
        at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message