ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Rakov <ivan.glu...@gmail.com>
Subject Re: Large durable caches
Date Mon, 19 Feb 2018 08:46:58 GMT
Hello,

Seems like checkpoint-runner thread is idle. There's no active 
checkpoint at the moment of dump, and thus there's no work for 
checkpoint thread.
If the problem persists, please share full thread dump. It would be 
better to take 2-3 consecutive thread dumps when your data streamer stops.

Best Regards,
Ivan Rakov

On 19.02.2018 0:41, lawrencefinn wrote:
> I am testing a very large durable cache that mostly cannot fit into memory.
> I start loading in a lot of data via a data streamer.  At some point the
> data becomes too large to fit into memory so ignite starts writing a lot to
> disk during checkpoints.  But after some point after that, the datastreamer
> stops streaming.  It's stuck.  Doing a jstack on ignite i see the
> datastreamer threads are all stuck in a parked state.  Any thoughts?
>
> "data-streamer-stripe-4-#37" #62 prio=5 os_prio=0 tid=0x00007fe8c212d000
> nid=0x1410 waiting on condition [0x00007fe73d1d6000]
>     java.lang.Thread.State: WAITING (parking)
>          at sun.misc.Unsafe.park(Native Method)
>          at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
>          at
> org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:651)
>          at
> org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499)
>          at java.lang.Thread.run(Thread.java:745)
>
> The checkpoint thread seems to be parked too
> "checkpoint-runner-#354" #397 prio=5 os_prio=0 tid=0x00007fe49c04f000
> nid=0x177c waiting on condition [0x00007fdc54a1c000]
>     java.lang.Thread.State: WAITING (parking)
>          at sun.misc.Unsafe.park(Native Method)
>          - parking to wait for  <0x00000002c08a6ef8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>          at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
>          at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
>          at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
>          at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
>          at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
>          at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>          at java.lang.Thread.run(Thread.java:745)
>
>     Locked ownable synchronizers:
>          - None
>
>
> My configuration is as follows:
>
>      <bean id="ignite.cfg"
> class="org.apache.ignite.configuration.IgniteConfiguration">
>      <property name="publicThreadPoolSize" value="128"/>
>      <property name="systemThreadPoolSize" value="128"/>
>      <property name="dataStreamerThreadPoolSize" value="128"/>
>      <property name="deploymentMode" value="SHARED"/>
>      <property name="peerClassLoadingEnabled" value="true"/>
>      <property name="dataStorageConfiguration">
>      <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>          <property name="storagePath" value="/per/persistence"/>
>          <property name="walPath" value="/wal"/>
>          <property name="walMode" value="BACKGROUND"/>
>          <property name="walFlushFrequency" value="10000"/>
>          <property name="checkpointThreads" value="128"/>
>          <property name="writeThrottlingEnabled" value="true"/>
>          <property name="walArchivePath" value="/wal/archive"/>
>          <property name="defaultDataRegionConfiguration">
>                  <bean
> class="org.apache.ignite.configuration.DataRegionConfiguration">
>                          <property name="name" value="Default_Region"/>
>                          <property name="maxSize" value="#{100L * 1024 *
> 1024}"/>
>                  </bean>
>          </property>
>          <property name="dataRegionConfigurations">
>              <list>
>                <bean
> class="org.apache.ignite.configuration.DataRegionConfiguration">
>                  <property name="name" value="20G_Region_Eviction_P"/>
>                  <property name="initialSize" value="#{10L * 1024 * 1024}"/>
>                  <property name="maxSize" value="#{20 * 1024L * 1024 *
> 1024}"/>
>                  <property name="persistenceEnabled" value="true"/>
>                  <property name="pageEvictionMode" value="RANDOM_2_LRU"/>
>                  <property name="checkpointPageBufferSize" value="#{2 * 1024L
> * 1024 * 1024}"/>
>                </bean>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/


Mime
View raw message