ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From waterg <jessie.jianwei....@gmail.com>
Subject Re: Disable WriteBehind
Date Thu, 13 Apr 2017 17:36:09 GMT
Hello Nikolai,

Thank you for your reply.
I'm working a simplified maven project, to reproduce.
Btw, with this configuration below, we did observed batch updatein
persistent store.

<property name="readThrough" value="true"/>
<property name="writeThrough" value="true"/>
<property name="writeBehindEnabled" value="true"/>
<property name="writeBehindFlushSize" value="499"/>
<property name="WriteBehindFlushFrequency" value="0"/>
<property name="writeBehindFlushThreadCount" value="1"/>
<property name="writeBehindBatchSize" value="500"/>

However as soon as we add the cache.remove() in,
we start to see the behavior changed to a lot of batch operations with a
few records.
Is there any reasons for this? Does cache.remove trigger flushing out to
persistent layer?
Thank you for your help!

[1492104394638]-----------Datebase BATCH upsert:1 entries successful
----------------
[1492104394772]-----------Datebase BATCH upsert:3 entries successful
----------------
[1492104395042]-----------Datebase BATCH upsert:1 entries successful
----------------
[1492104395170]-----------Datebase BATCH DELETE:1 entries successful
----------------
[1492104395452]-----------Datebase BATCH upsert:1 entries successful
----------------
[1492104395587]-----------Datebase BATCH upsert:1 entries successful
----------------


On Tue, Apr 11, 2017 at 4:39 AM, Nikolai Tikhonov-2 [via Apache Ignite
Users] <ml-node+s70518n11878h66@n6.nabble.com> wrote:

> > If I disable writeThrough, would a put operation on the cache still
> succeed?
> Yes, sure. If write Through enabled than entries will be propagated to
> underlying store too.
>
> > If so, the get operation would return the same result as if the
> writeThrough were enabled, correct?
> You're right. But if you configure expire or eviction policy then a get
> operations might be miss.
>
> Could you share simple maven project which can reproduce the behaviour?
>
> On Mon, Apr 10, 2017 at 9:59 PM, waterg <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=11878&i=0>> wrote:
>
>> Thank you for reply Nikolai. I have a more complex nested if-else logic
>> than the condition 1 and condition 2 here. They are based on the results of
>> SQLQueries from cache only. We don't use any conditions based on querying
>> persistent store. These two are examples of different put and other
>> operations may happen based on what conditions are met.
>>
>> If I disable writeThrough, would a put operation on the cache still
>> succeed? If so, the get operation would return the same result as if the
>> writeThrough were enabled, correct?
>>
>>
>>
>>
>> On Mon, Apr 10, 2017 at 9:53 AM, Nikolai Tikhonov-2 [via Apache Ignite
>> Users] <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=11862&i=0>> wrote:
>>
>>> What did you mean behind condition1 and condition2? Might be you have
>>> case when you have more "miss" in access to entries? For example you
>>> disable writeThrought and after it an get operations return null in the
>>> most cases and you make more complex logic.
>>>
>>> On Fri, Apr 7, 2017 at 7:43 PM, waterg <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=11858&i=0>> wrote:
>>>
>>>>
>>>> The entry point looks like this
>>>>
>>>>    try(Ignite ignite = Ignition.start(getConfigFile())) {
>>>>
>>>>             IgniteTransactions txs = ignite.transactions();
>>>>             IgniteCache<String, StagingRec> stagingCache = ignite.getOrCreateCache("stagingCache");
>>>>             IgniteCache<String, TargeRec> targetCache = ignite.getOrCreateCache("targetCache");
>>>>             //Sequence number for guid
>>>>             IgniteAtomicSequence guidSeq = getGuidSeq(ignite, targetCache);
>>>>             applicationService service  = new applicationService(targetCache,
guidSeq);
>>>>             //load staging
>>>>             loadStaging(stagingCache);
>>>>
>>>>             //process staging data
>>>>             SqlQuery<String,stagingRec> sqlStg = new SqlQuery<>(StagingRec.class,
getStagingSql());
>>>>             try (QueryCursor<Cache.Entry<String, StagingRec>>
cursor = stagingCache.query(sqlStg)) {
>>>>
>>>>                 for (Cache.Entry<String, StagingRec> e : cursor) {
>>>>                     Transaction tx = txs.txStart(TransactionConcurrency.PESSIMISTIC,
TransactionIsolation.REPEATABLE_READ);
>>>>                     service.processStaging(e);
>>>> //                    stagingCache.remove(e.getKey()); //remove entry from
staging
>>>>                     tx.commit();
>>>>                 }
>>>>             }
>>>>         }catch (Exception e) {
>>>>             e.printStackTrace();
>>>>         }
>>>>
>>>> In service.processStaging, the logic looks like this:
>>>>
>>>> if (condition1) {
>>>>       targetCache.put(key, value);
>>>> } else if (condition2) {
>>>>       targetCache.remove(key, value);
>>>>       targetCache.put(key2, value2);
>>>> }
>>>>
>>>> Do you see anything that might be causing the issue?
>>>>
>>>> On Fri, Apr 7, 2017 at 2:26 AM, Nikolai Tikhonov-2 [via Apache Ignite
>>>> Users] <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=11827&i=0>> wrote:
>>>>
>>>>> Could you share code snippet which your benchmarked?
>>>>>
>>>>> On Fri, Apr 7, 2017 at 6:03 AM, waterg <[hidden email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=11799&i=0>>
wrote:
>>>>>
>>>>>> I have two caches. The application takes the first cache as input
and
>>>>>> output value to the second cache.
>>>>>>
>>>>>> The first cache has readThrough only.
>>>>>>
>>>>>> Part of the configurations for second caches are below:
>>>>>>
>>>>>> <!-- Enable readThrough-->
>>>>>> <property name="readThrough" value="true"/>
>>>>>> <property name="writeThrough" value="true"/>
>>>>>> <property name="writeBehindEnabled" value="true"/>
>>>>>> <property name="writeBehindFlushSize" value="499"/>
>>>>>> <property name="WriteBehindFlushFrequency" value="0"/>
>>>>>> <property name="writeBehindFlushThreadCount" value="1"/>
>>>>>> <property name="writeBehindBatchSize" value="500"/>
>>>>>>
>>>>>> There are also two indexes on this cache.
>>>>>>
>>>>>> The other case is to set writeThrough and writeBehindEnabled to
>>>>>> false. I didn't change other settings.
>>>>>> Is there anything else that might be relevant to this case?
>>>>>>
>>>>>> Best,
>>>>>>
>>>>>>
>>>>>> On Thu, Apr 6, 2017 at 10:33 AM, Nikolai Tikhonov-2 [via Apache
>>>>>> Ignite Users] <[hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node&node=11792&i=0>>
wrote:
>>>>>>
>>>>>>> It's strange. Could you share your configuration for both case?
Also
>>>>>>> could you describe more your case?
>>>>>>>
>>>>>>> On Wed, Apr 5, 2017 at 8:45 PM, waterg <[hidden email]
>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11789&i=0>>
wrote:
>>>>>>>
>>>>>>>> Thank you for the reply.
>>>>>>>> Yes, I disabled both write through and write behind.
>>>>>>>> We're trying evaluate the application's performance on ignite
and
>>>>>>>> by taking
>>>>>>>> the persistent store out of equation, we thought the performance
>>>>>>>> shall
>>>>>>>> improve, but on the contrary we saw performance dropped over
30%.
>>>>>>>> What would
>>>>>>>> explain this kind of behavior?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context: http://apache-ignite-users.705
>>>>>>>> 18.x6.nabble.com/Disable-WriteBehind-tp11729p11763.html
>>>>>>>> Sent from the Apache Ignite Users mailing list archive at
>>>>>>>> Nabble.com.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> If you reply to this email, your message will be added to the
>>>>>>> discussion below:
>>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>>>> Behind-tp11729p11789.html
>>>>>>> To unsubscribe from Disable WriteBehind, click here.
>>>>>>> NAML
>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> View this message in context: Re: Disable WriteBehind
>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11792.html>
>>>>>>
>>>>>> Sent from the Apache Ignite Users mailing list archive
>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> If you reply to this email, your message will be added to the
>>>>> discussion below:
>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>> Behind-tp11729p11799.html
>>>>> To unsubscribe from Disable WriteBehind, click here.
>>>>> NAML
>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> View this message in context: Re: Disable WriteBehind
>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11827.html>
>>>> Sent from the Apache Ignite Users mailing list archive
>>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>>
>>>
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>> Behind-tp11729p11858.html
>>> To unsubscribe from Disable WriteBehind, click here.
>>> NAML
>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>
>>
>> ------------------------------
>> View this message in context: Re: Disable WriteBehind
>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11862.html>
>> Sent from the Apache Ignite Users mailing list archive
>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Disable-
> WriteBehind-tp11729p11878.html
> To unsubscribe from Disable WriteBehind, click here
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=11729&code=amVzc2llLmppYW53ZWkubGluQGdtYWlsLmNvbXwxMTcyOXwtOTM2MTAxMjg=>
> .
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11960.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Mime
View raw message