geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dinesh Akhand <dines...@amdocs.com>
Subject RE: Thread block on org.apache.geode.cache.CacheFactory.getAnyInstance(CacheFactory.java:282)
Date Wed, 24 Oct 2018 08:27:22 GMT
No dead local was there , reader thread are waiting mostly.
PFA whole stack strace.

Thanks,
Dinesh Akhand

-----Original Message-----
From: Anthony Baker <abaker@pivotal.io> 
Sent: Tuesday, October 23, 2018 7:34 PM
To: dev@geode.apache.org
Cc: user@geode.apache.org
Subject: Re: Thread block on org.apache.geode.cache.CacheFactory.getAnyInstance(CacheFactory.java:282)

Dinesh, have you analyzed the full thread dump to see if there is a deadlock?  I can’t tell
just from these 2 threads if there is a deadlock.

Anthony


> On Oct 23, 2018, at 6:32 AM, Dinesh Akhand <dineshak@amdocs.com> wrote:
> 
> Hi team,
> 
> Recently we see JVM stuck , in stack trace I can see below method 
> having problem
> 
> As per document
> 
>                From link :
> 
>                                
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/cach
> e/CacheListener.html
> 
>                this warning:
> 
> WARNING: To avoid risk of deadlock, do not invoke 
> CacheFactory.getAnyInstance() from within any callback methods. 
> Instead use EntryEvent.getRegion().getCache() or 
> RegionEvent.getRegion().getCache()
> What  is the best solution to avoid it.
> 
> Function Execution Processor1" #247 daemon prio=10 os_prio=0 tid=0x00007f5798268000 nid=0x3ff5
waiting for monitor entry [0x00007f576adf0000]
>   java.lang.Thread.State: BLOCKED (on object monitor)
>        at org.apache.geode.cache.CacheFactory.getAnyInstance(CacheFactory.java:282)
>        - waiting to lock <0x0000000699feafa0> (a java.lang.Class for org.apache.geode.cache.CacheFactory)
>        at org.apache.geode.management.internal.cli.functions.GetRegionsFunction.execute(GetRegionsFunction.java:44)
>        at org.apache.geode.internal.cache.MemberFunctionStreamingMessage.process(MemberFunctionStreamingMessage.java:185)
>        at org.apache.geode.distributed.internal.DistributionMessage.scheduleAction(DistributionMessage.java:374)
>        at org.apache.geode.distributed.internal.DistributionMessage$1.run(DistributionMessage.java:440)
>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>        at org.apache.geode.distributed.internal.DistributionManager.runUntilShutdown(DistributionManager.java:662)
>        at org.apache.geode.distributed.internal.DistributionManager$9$1.run(DistributionManager.java:1108)
>        at java.lang.Thread.run(Thread.java:745
> 
> "P2P message reader for 
> 10.218.110.61(sbimgapp16-server1:65602)<v7>:1026 shared ordered uid=139 port=62033"
#403 daemon prio=10 os_prio=0 tid=0x00007f1ad4114800 nid=0xda7c waiting for monitor entry
[0x0000 7f1a28fcc000]
>   java.lang.Thread.State: BLOCKED (on object monitor)
>        at org.apache.geode.cache.CacheFactory.getAnyInstance(CacheFactory.java:282)
>        - waiting to lock <0x000000021df685a8> (a java.lang.Class for org.apache.geode.cache.CacheFactory)
>        at amdocs.imdg.statistics.GemFireStatisticsFactory.getStatisticsFactory(GemFireStatisticsFactory.java:43)
>        at amdocs.imdg.statistics.VSDCountersManager.<init>(VSDCountersManager.java:35)
>        at amdocs.imdg.statistics.VSDCountersManager.<clinit>(VSDCountersManager.java:19)
>        at amdocs.imdg.statistics.CountersManagerFactory.getCountersManager(CountersManagerFactory.java:27)
>        at amdocs.imdg.utils.pooling.DataPoolFactory.makeObject(DataPoolFactory.java:42)
>        at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:797)
>        - locked <0x000000021ebbe630> (a org.apache.commons.pool.impl.GenericKeyedObjectPool)
>        at amdocs.imdg.utils.pooling.DataPool$DataPoolManager.getByteArray(DataPool.java:236)
>        at amdocs.imdg.utils.pooling.DataPool.getByteArray(DataPool.java:98)
>        at amdocs.imdg.model.BusinessData.populateData(BusinessData.java:110)
>        at amdocs.imdg.utils.FlatBuffersUtils.updateBusinessData(FlatBuffersUtils.java:2255)
>        at amdocs.imdg.utils.FlatBuffersUtils.updateCustomerData(FlatBuffersUtils.java:3083)
>        at amdocs.imdg.utils.FlatBuffersUtils.updateNewCustomer(FlatBuffersUtils.java:3103)
>        at amdocs.imdg.utils.FlatBuffersUtils.updateFromCustomerData(FlatBuffersUtils.java:2798)
>        at amdocs.imdg.model.Customer.fromData(Customer.java:696)
>        - locked <0x000000021f874d08> (a amdocs.imdg.model.Customer)
>        at org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2372)
>        at org.apache.geode.internal.InternalDataSerializer.readDataSerializable(InternalDataSerializer.java:2395)
>        at org.apache.geode.internal.InternalDataSerializer.basicRead
> 
> 
> 
> Thanks,
> Dinesh Akhand
> 
> “Amdocs’ email platform is based on a third-party, worldwide, cloud-based system.
Any emails sent to Amdocs will be processed and stored using such system and are accessible
by third party providers of such system on a limited basis. Your sending of emails to Amdocs
evidences your consent to the use of such system and such processing, storing and access”.

“Amdocs’ email platform is based on a third-party, worldwide, cloud-based system. Any
emails sent to Amdocs will be processed and stored using such system and are accessible by
third party providers of such system on a limited basis. Your sending of emails to Amdocs
evidences your consent to the use of such system and such processing, storing and access”.
Mime
View raw message