ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nikolai Tikhonov <ntikho...@apache.org>
Subject Re: Disable WriteBehind
Date Fri, 14 Apr 2017 16:07:02 GMT
Yes, Ignite.log.

On Fri, Apr 14, 2017 at 6:54 PM, waterg <jessie.jianwei.lin@gmail.com>
wrote:

> Hi Nikolai, No topology changes. We have 2 server nodes and a client node.
>
> What kind of log you refer to? Ignite log?
>
> Jessie
>
> On Fri, Apr 14, 2017 at 6:52 AM, Nikolai Tikhonov-2 [via Apache Ignite
> Users] <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=11983&i=0>> wrote:
>
>> Hello!
>>
>> Was topology stable? Could you share full logs for this case?
>>
>> On Thu, Apr 13, 2017 at 8:36 PM, waterg <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=11979&i=0>> wrote:
>>
>>> Hello Nikolai,
>>>
>>> Thank you for your reply.
>>> I'm working a simplified maven project, to reproduce.
>>> Btw, with this configuration below, we did observed batch updatein
>>> persistent store.
>>>
>>> <property name="readThrough" value="true"/>
>>> <property name="writeThrough" value="true"/>
>>> <property name="writeBehindEnabled" value="true"/>
>>> <property name="writeBehindFlushSize" value="499"/>
>>> <property name="WriteBehindFlushFrequency" value="0"/>
>>> <property name="writeBehindFlushThreadCount" value="1"/>
>>> <property name="writeBehindBatchSize" value="500"/>
>>>
>>> However as soon as we add the cache.remove() in,
>>> we start to see the behavior changed to a lot of batch operations with a
>>> few records.
>>> Is there any reasons for this? Does cache.remove trigger flushing out to
>>> persistent layer?
>>> Thank you for your help!
>>>
>>> [1492104394638]-----------Datebase BATCH upsert:1 entries successful
>>> ----------------
>>> [1492104394772]-----------Datebase BATCH upsert:3 entries successful
>>> ----------------
>>> [1492104395042]-----------Datebase BATCH upsert:1 entries successful
>>> ----------------
>>> [1492104395170]-----------Datebase BATCH DELETE:1 entries successful
>>> ----------------
>>> [1492104395452]-----------Datebase BATCH upsert:1 entries successful
>>> ----------------
>>> [1492104395587]-----------Datebase BATCH upsert:1 entries successful
>>> ----------------
>>>
>>>
>>> On Tue, Apr 11, 2017 at 4:39 AM, Nikolai Tikhonov-2 [via Apache Ignite
>>> Users] <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=11960&i=0>> wrote:
>>>
>>>> > If I disable writeThrough, would a put operation on the cache still
>>>> succeed?
>>>> Yes, sure. If write Through enabled than entries will be propagated to
>>>> underlying store too.
>>>>
>>>> > If so, the get operation would return the same result as if the
>>>> writeThrough were enabled, correct?
>>>> You're right. But if you configure expire or eviction policy then a get
>>>> operations might be miss.
>>>>
>>>> Could you share simple maven project which can reproduce the behaviour?
>>>>
>>>> On Mon, Apr 10, 2017 at 9:59 PM, waterg <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=11878&i=0>> wrote:
>>>>
>>>>> Thank you for reply Nikolai. I have a more complex nested if-else
>>>>> logic than the condition 1 and condition 2 here. They are based on the
>>>>> results of SQLQueries from cache only. We don't use any conditions based
on
>>>>> querying persistent store. These two are examples of different put and
>>>>> other operations may happen based on what conditions are met.
>>>>>
>>>>> If I disable writeThrough, would a put operation on the cache still
>>>>> succeed? If so, the get operation would return the same result as if
the
>>>>> writeThrough were enabled, correct?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Apr 10, 2017 at 9:53 AM, Nikolai Tikhonov-2 [via Apache Ignite
>>>>> Users] <[hidden email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=11862&i=0>>
wrote:
>>>>>
>>>>>> What did you mean behind condition1 and condition2? Might be you
have
>>>>>> case when you have more "miss" in access to entries? For example
you
>>>>>> disable writeThrought and after it an get operations return null
in the
>>>>>> most cases and you make more complex logic.
>>>>>>
>>>>>> On Fri, Apr 7, 2017 at 7:43 PM, waterg <[hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node&node=11858&i=0>>
wrote:
>>>>>>
>>>>>>>
>>>>>>> The entry point looks like this
>>>>>>>
>>>>>>>    try(Ignite ignite = Ignition.start(getConfigFile())) {
>>>>>>>
>>>>>>>             IgniteTransactions txs = ignite.transactions();
>>>>>>>             IgniteCache<String, StagingRec> stagingCache
= ignite.getOrCreateCache("stagingCache");
>>>>>>>             IgniteCache<String, TargeRec> targetCache =
ignite.getOrCreateCache("targetCache");
>>>>>>>             //Sequence number for guid
>>>>>>>             IgniteAtomicSequence guidSeq = getGuidSeq(ignite,
targetCache);
>>>>>>>             applicationService service  = new applicationService(targetCache,
guidSeq);
>>>>>>>             //load staging
>>>>>>>             loadStaging(stagingCache);
>>>>>>>
>>>>>>>             //process staging data
>>>>>>>             SqlQuery<String,stagingRec> sqlStg = new SqlQuery<>(StagingRec.class,
getStagingSql());
>>>>>>>             try (QueryCursor<Cache.Entry<String, StagingRec>>
cursor = stagingCache.query(sqlStg)) {
>>>>>>>
>>>>>>>                 for (Cache.Entry<String, StagingRec> e
: cursor) {
>>>>>>>                     Transaction tx = txs.txStart(TransactionConcurrency.PESSIMISTIC,
TransactionIsolation.REPEATABLE_READ);
>>>>>>>                     service.processStaging(e);
>>>>>>> //                    stagingCache.remove(e.getKey()); //remove
entry from staging
>>>>>>>                     tx.commit();
>>>>>>>                 }
>>>>>>>             }
>>>>>>>         }catch (Exception e) {
>>>>>>>             e.printStackTrace();
>>>>>>>         }
>>>>>>>
>>>>>>> In service.processStaging, the logic looks like this:
>>>>>>>
>>>>>>> if (condition1) {
>>>>>>>       targetCache.put(key, value);
>>>>>>> } else if (condition2) {
>>>>>>>       targetCache.remove(key, value);
>>>>>>>       targetCache.put(key2, value2);
>>>>>>> }
>>>>>>>
>>>>>>> Do you see anything that might be causing the issue?
>>>>>>>
>>>>>>> On Fri, Apr 7, 2017 at 2:26 AM, Nikolai Tikhonov-2 [via Apache
>>>>>>> Ignite Users] <[hidden email]
>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11827&i=0>>
wrote:
>>>>>>>
>>>>>>>> Could you share code snippet which your benchmarked?
>>>>>>>>
>>>>>>>> On Fri, Apr 7, 2017 at 6:03 AM, waterg <[hidden email]
>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11799&i=0>>
wrote:
>>>>>>>>
>>>>>>>>> I have two caches. The application takes the first cache
as input
>>>>>>>>> and output value to the second cache.
>>>>>>>>>
>>>>>>>>> The first cache has readThrough only.
>>>>>>>>>
>>>>>>>>> Part of the configurations for second caches are below:
>>>>>>>>>
>>>>>>>>> <!-- Enable readThrough-->
>>>>>>>>> <property name="readThrough" value="true"/>
>>>>>>>>> <property name="writeThrough" value="true"/>
>>>>>>>>> <property name="writeBehindEnabled" value="true"/>
>>>>>>>>> <property name="writeBehindFlushSize" value="499"/>
>>>>>>>>> <property name="WriteBehindFlushFrequency" value="0"/>
>>>>>>>>> <property name="writeBehindFlushThreadCount" value="1"/>
>>>>>>>>> <property name="writeBehindBatchSize" value="500"/>
>>>>>>>>>
>>>>>>>>> There are also two indexes on this cache.
>>>>>>>>>
>>>>>>>>> The other case is to set writeThrough and writeBehindEnabled
to
>>>>>>>>> false. I didn't change other settings.
>>>>>>>>> Is there anything else that might be relevant to this
case?
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Apr 6, 2017 at 10:33 AM, Nikolai Tikhonov-2 [via
Apache
>>>>>>>>> Ignite Users] <[hidden email]
>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11792&i=0>>
wrote:
>>>>>>>>>
>>>>>>>>>> It's strange. Could you share your configuration
for both case?
>>>>>>>>>> Also could you describe more your case?
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 5, 2017 at 8:45 PM, waterg <[hidden
email]
>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11789&i=0>>
wrote:
>>>>>>>>>>
>>>>>>>>>>> Thank you for the reply.
>>>>>>>>>>> Yes, I disabled both write through and write
behind.
>>>>>>>>>>> We're trying evaluate the application's performance
on ignite
>>>>>>>>>>> and by taking
>>>>>>>>>>> the persistent store out of equation, we thought
the performance
>>>>>>>>>>> shall
>>>>>>>>>>> improve, but on the contrary we saw performance
dropped over
>>>>>>>>>>> 30%. What would
>>>>>>>>>>> explain this kind of behavior?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context: http://apache-ignite-users.705
>>>>>>>>>>> 18.x6.nabble.com/Disable-WriteBehind-tp11729p11763.html
>>>>>>>>>>> Sent from the Apache Ignite Users mailing list
archive at
>>>>>>>>>>> Nabble.com.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------
>>>>>>>>>> If you reply to this email, your message will be
added to the
>>>>>>>>>> discussion below:
>>>>>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>>>>>>> Behind-tp11729p11789.html
>>>>>>>>>> To unsubscribe from Disable WriteBehind, click here.
>>>>>>>>>> NAML
>>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------
>>>>>>>>> View this message in context: Re: Disable WriteBehind
>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11792.html>
>>>>>>>>>
>>>>>>>>> Sent from the Apache Ignite Users mailing list archive
>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/>
at Nabble.com.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>> If you reply to this email, your message will be added to
the
>>>>>>>> discussion below:
>>>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>>>>> Behind-tp11729p11799.html
>>>>>>>> To unsubscribe from Disable WriteBehind, click here.
>>>>>>>> NAML
>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> View this message in context: Re: Disable WriteBehind
>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11827.html>
>>>>>>> Sent from the Apache Ignite Users mailing list archive
>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> If you reply to this email, your message will be added to the
>>>>>> discussion below:
>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>>> Behind-tp11729p11858.html
>>>>>> To unsubscribe from Disable WriteBehind, click here.
>>>>>> NAML
>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> View this message in context: Re: Disable WriteBehind
>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11862.html>
>>>>> Sent from the Apache Ignite Users mailing list archive
>>>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>>>
>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> If you reply to this email, your message will be added to the
>>>> discussion below:
>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>> Behind-tp11729p11878.html
>>>> To unsubscribe from Disable WriteBehind, click here.
>>>> NAML
>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>
>>>
>>>
>>> ------------------------------
>>> View this message in context: Re: Disable WriteBehind
>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11960.html>
>>> Sent from the Apache Ignite Users mailing list archive
>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>
>>
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>> Behind-tp11729p11979.html
>> To unsubscribe from Disable WriteBehind, click here.
>> NAML
>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ------------------------------
> View this message in context: Re: Disable WriteBehind
> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11983.html>
> Sent from the Apache Ignite Users mailing list archive
> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>

Mime
View raw message