ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From waterg <jessie.jianwei....@gmail.com>
Subject Re: Disable WriteBehind
Date Tue, 18 Apr 2017 22:43:18 GMT
Hi Nikolai, looks like that was a wrong log file.
I have reran the app and here's a new log file.
Appreciate your help.

Jessie

On Mon, Apr 17, 2017 at 3:01 AM, Nikolai Tikhonov-2 [via Apache Ignite
Users] <ml-node+s70518n11993h31@n6.nabble.com> wrote:

> Hi,
>
> I see in logs that node lives 10 seconds and you have configured only
> system caches. Did you run you test on this setup?
>
> On Sat, Apr 15, 2017 at 2:42 AM, waterg <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=11993&i=0>> wrote:
>
>> Hi, please see the log attached.
>>
>> On Fri, Apr 14, 2017 at 9:07 AM, Nikolai Tikhonov-2 [via Apache Ignite
>> Users] <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=11987&i=0>> wrote:
>>
>>> Yes, Ignite.log.
>>>
>>> On Fri, Apr 14, 2017 at 6:54 PM, waterg <[hidden email]
>>> <http:///user/SendEmail.jtp?type=node&node=11984&i=0>> wrote:
>>>
>>>> Hi Nikolai, No topology changes. We have 2 server nodes and a client
>>>> node.
>>>>
>>>> What kind of log you refer to? Ignite log?
>>>>
>>>> Jessie
>>>>
>>>> On Fri, Apr 14, 2017 at 6:52 AM, Nikolai Tikhonov-2 [via Apache Ignite
>>>> Users] <[hidden email]
>>>> <http:///user/SendEmail.jtp?type=node&node=11983&i=0>> wrote:
>>>>
>>>>> Hello!
>>>>>
>>>>> Was topology stable? Could you share full logs for this case?
>>>>>
>>>>> On Thu, Apr 13, 2017 at 8:36 PM, waterg <[hidden email]
>>>>> <http:///user/SendEmail.jtp?type=node&node=11979&i=0>>
wrote:
>>>>>
>>>>>> Hello Nikolai,
>>>>>>
>>>>>> Thank you for your reply.
>>>>>> I'm working a simplified maven project, to reproduce.
>>>>>> Btw, with this configuration below, we did observed batch updatein
>>>>>> persistent store.
>>>>>>
>>>>>> <property name="readThrough" value="true"/>
>>>>>> <property name="writeThrough" value="true"/>
>>>>>> <property name="writeBehindEnabled" value="true"/>
>>>>>> <property name="writeBehindFlushSize" value="499"/>
>>>>>> <property name="WriteBehindFlushFrequency" value="0"/>
>>>>>> <property name="writeBehindFlushThreadCount" value="1"/>
>>>>>> <property name="writeBehindBatchSize" value="500"/>
>>>>>>
>>>>>> However as soon as we add the cache.remove() in,
>>>>>> we start to see the behavior changed to a lot of batch operations
>>>>>> with a few records.
>>>>>> Is there any reasons for this? Does cache.remove trigger flushing
out
>>>>>> to persistent layer?
>>>>>> Thank you for your help!
>>>>>>
>>>>>> [1492104394638]-----------Datebase BATCH upsert:1 entries successful
>>>>>> ----------------
>>>>>> [1492104394772]-----------Datebase BATCH upsert:3 entries successful
>>>>>> ----------------
>>>>>> [1492104395042]-----------Datebase BATCH upsert:1 entries successful
>>>>>> ----------------
>>>>>> [1492104395170]-----------Datebase BATCH DELETE:1 entries successful
>>>>>> ----------------
>>>>>> [1492104395452]-----------Datebase BATCH upsert:1 entries successful
>>>>>> ----------------
>>>>>> [1492104395587]-----------Datebase BATCH upsert:1 entries successful
>>>>>> ----------------
>>>>>>
>>>>>>
>>>>>> On Tue, Apr 11, 2017 at 4:39 AM, Nikolai Tikhonov-2 [via Apache
>>>>>> Ignite Users] <[hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node&node=11960&i=0>>
wrote:
>>>>>>
>>>>>>> > If I disable writeThrough, would a put operation on the
cache
>>>>>>> still succeed?
>>>>>>> Yes, sure. If write Through enabled than entries will be propagated
>>>>>>> to underlying store too.
>>>>>>>
>>>>>>> > If so, the get operation would return the same result as
if the
>>>>>>> writeThrough were enabled, correct?
>>>>>>> You're right. But if you configure expire or eviction policy
then a
>>>>>>> get operations might be miss.
>>>>>>>
>>>>>>> Could you share simple maven project which can reproduce the
>>>>>>> behaviour?
>>>>>>>
>>>>>>> On Mon, Apr 10, 2017 at 9:59 PM, waterg <[hidden email]
>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11878&i=0>>
wrote:
>>>>>>>
>>>>>>>> Thank you for reply Nikolai. I have a more complex nested
if-else
>>>>>>>> logic than the condition 1 and condition 2 here. They are
based on the
>>>>>>>> results of SQLQueries from cache only. We don't use any conditions
based on
>>>>>>>> querying persistent store. These two are examples of different
put and
>>>>>>>> other operations may happen based on what conditions are
met.
>>>>>>>>
>>>>>>>> If I disable writeThrough, would a put operation on the cache
still
>>>>>>>> succeed? If so, the get operation would return the same result
as if the
>>>>>>>> writeThrough were enabled, correct?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 10, 2017 at 9:53 AM, Nikolai Tikhonov-2 [via
Apache
>>>>>>>> Ignite Users] <[hidden email]
>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11862&i=0>>
wrote:
>>>>>>>>
>>>>>>>>> What did you mean behind condition1 and condition2? Might
be you
>>>>>>>>> have case when you have more "miss" in access to entries?
For example you
>>>>>>>>> disable writeThrought and after it an get operations
return null in the
>>>>>>>>> most cases and you make more complex logic.
>>>>>>>>>
>>>>>>>>> On Fri, Apr 7, 2017 at 7:43 PM, waterg <[hidden email]
>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11858&i=0>>
wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The entry point looks like this
>>>>>>>>>>
>>>>>>>>>>    try(Ignite ignite = Ignition.start(getConfigFile()))
{
>>>>>>>>>>
>>>>>>>>>>             IgniteTransactions txs = ignite.transactions();
>>>>>>>>>>             IgniteCache<String, StagingRec>
stagingCache = ignite.getOrCreateCache("stagingCache");
>>>>>>>>>>             IgniteCache<String, TargeRec> targetCache
= ignite.getOrCreateCache("targetCache");
>>>>>>>>>>             //Sequence number for guid
>>>>>>>>>>             IgniteAtomicSequence guidSeq = getGuidSeq(ignite,
targetCache);
>>>>>>>>>>             applicationService service  = new applicationService(targetCache,
guidSeq);
>>>>>>>>>>             //load staging
>>>>>>>>>>             loadStaging(stagingCache);
>>>>>>>>>>
>>>>>>>>>>             //process staging data
>>>>>>>>>>             SqlQuery<String,stagingRec> sqlStg
= new SqlQuery<>(StagingRec.class, getStagingSql());
>>>>>>>>>>             try (QueryCursor<Cache.Entry<String,
StagingRec>> cursor = stagingCache.query(sqlStg)) {
>>>>>>>>>>
>>>>>>>>>>                 for (Cache.Entry<String, StagingRec>
e : cursor) {
>>>>>>>>>>                     Transaction tx = txs.txStart(TransactionConcurrency.PESSIMISTIC,
TransactionIsolation.REPEATABLE_READ);
>>>>>>>>>>                     service.processStaging(e);
>>>>>>>>>> //                    stagingCache.remove(e.getKey());
//remove entry from staging
>>>>>>>>>>                     tx.commit();
>>>>>>>>>>                 }
>>>>>>>>>>             }
>>>>>>>>>>         }catch (Exception e) {
>>>>>>>>>>             e.printStackTrace();
>>>>>>>>>>         }
>>>>>>>>>>
>>>>>>>>>> In service.processStaging, the logic looks like this:
>>>>>>>>>>
>>>>>>>>>> if (condition1) {
>>>>>>>>>>       targetCache.put(key, value);
>>>>>>>>>> } else if (condition2) {
>>>>>>>>>>       targetCache.remove(key, value);
>>>>>>>>>>       targetCache.put(key2, value2);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Do you see anything that might be causing the issue?
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 7, 2017 at 2:26 AM, Nikolai Tikhonov-2
[via Apache
>>>>>>>>>> Ignite Users] <[hidden email]
>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11827&i=0>>
wrote:
>>>>>>>>>>
>>>>>>>>>>> Could you share code snippet which your benchmarked?
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Apr 7, 2017 at 6:03 AM, waterg <[hidden
email]
>>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11799&i=0>>
wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I have two caches. The application takes
the first cache as
>>>>>>>>>>>> input and output value to the second cache.
>>>>>>>>>>>>
>>>>>>>>>>>> The first cache has readThrough only.
>>>>>>>>>>>>
>>>>>>>>>>>> Part of the configurations for second caches
are below:
>>>>>>>>>>>>
>>>>>>>>>>>> <!-- Enable readThrough-->
>>>>>>>>>>>> <property name="readThrough" value="true"/>
>>>>>>>>>>>> <property name="writeThrough" value="true"/>
>>>>>>>>>>>> <property name="writeBehindEnabled" value="true"/>
>>>>>>>>>>>> <property name="writeBehindFlushSize"
value="499"/>
>>>>>>>>>>>> <property name="WriteBehindFlushFrequency"
value="0"/>
>>>>>>>>>>>> <property name="writeBehindFlushThreadCount"
value="1"/>
>>>>>>>>>>>> <property name="writeBehindBatchSize"
value="500"/>
>>>>>>>>>>>>
>>>>>>>>>>>> There are also two indexes on this cache.
>>>>>>>>>>>>
>>>>>>>>>>>> The other case is to set writeThrough and
writeBehindEnabled to
>>>>>>>>>>>> false. I didn't change other settings.
>>>>>>>>>>>> Is there anything else that might be relevant
to this case?
>>>>>>>>>>>>
>>>>>>>>>>>> Best,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Apr 6, 2017 at 10:33 AM, Nikolai
Tikhonov-2 [via Apache
>>>>>>>>>>>> Ignite Users] <[hidden email]
>>>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11792&i=0>>
wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> It's strange. Could you share your configuration
for both
>>>>>>>>>>>>> case? Also could you describe more your
case?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Apr 5, 2017 at 8:45 PM, waterg
<[hidden email]
>>>>>>>>>>>>> <http:///user/SendEmail.jtp?type=node&node=11789&i=0>>
wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you for the reply.
>>>>>>>>>>>>>> Yes, I disabled both write through
and write behind.
>>>>>>>>>>>>>> We're trying evaluate the application's
performance on ignite
>>>>>>>>>>>>>> and by taking
>>>>>>>>>>>>>> the persistent store out of equation,
we thought the
>>>>>>>>>>>>>> performance shall
>>>>>>>>>>>>>> improve, but on the contrary we saw
performance dropped over
>>>>>>>>>>>>>> 30%. What would
>>>>>>>>>>>>>> explain this kind of behavior?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> View this message in context: http://apache-ignite-users.705
>>>>>>>>>>>>>> 18.x6.nabble.com/Disable-WriteBehind-tp11729p11763.html
>>>>>>>>>>>>>> Sent from the Apache Ignite Users
mailing list archive at
>>>>>>>>>>>>>> Nabble.com.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>> If you reply to this email, your message
will be added to the
>>>>>>>>>>>>> discussion below:
>>>>>>>>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>>>>>>>>>> Behind-tp11729p11789.html
>>>>>>>>>>>>> To unsubscribe from Disable WriteBehind,
click here.
>>>>>>>>>>>>> NAML
>>>>>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>> View this message in context: Re: Disable
WriteBehind
>>>>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11792.html>
>>>>>>>>>>>>
>>>>>>>>>>>> Sent from the Apache Ignite Users mailing
list archive
>>>>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/>
at
>>>>>>>>>>>> Nabble.com.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------
>>>>>>>>>>> If you reply to this email, your message will
be added to the
>>>>>>>>>>> discussion below:
>>>>>>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>>>>>>>> Behind-tp11729p11799.html
>>>>>>>>>>> To unsubscribe from Disable WriteBehind, click
here.
>>>>>>>>>>> NAML
>>>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------
>>>>>>>>>> View this message in context: Re: Disable WriteBehind
>>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11827.html>
>>>>>>>>>> Sent from the Apache Ignite Users mailing list archive
>>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/>
at Nabble.com.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------
>>>>>>>>> If you reply to this email, your message will be added
to the
>>>>>>>>> discussion below:
>>>>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>>>>>> Behind-tp11729p11858.html
>>>>>>>>> To unsubscribe from Disable WriteBehind, click here.
>>>>>>>>> NAML
>>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>> View this message in context: Re: Disable WriteBehind
>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11862.html>
>>>>>>>> Sent from the Apache Ignite Users mailing list archive
>>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/> at
Nabble.com.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>> If you reply to this email, your message will be added to the
>>>>>>> discussion below:
>>>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>>>> Behind-tp11729p11878.html
>>>>>>> To unsubscribe from Disable WriteBehind, click here.
>>>>>>> NAML
>>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>> View this message in context: Re: Disable WriteBehind
>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11960.html>
>>>>>> Sent from the Apache Ignite Users mailing list archive
>>>>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> If you reply to this email, your message will be added to the
>>>>> discussion below:
>>>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>>>> Behind-tp11729p11979.html
>>>>> To unsubscribe from Disable WriteBehind, click here.
>>>>> NAML
>>>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>>>
>>>>
>>>>
>>>> ------------------------------
>>>> View this message in context: Re: Disable WriteBehind
>>>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11983.html>
>>>> Sent from the Apache Ignite Users mailing list archive
>>>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>>>
>>>
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>> http://apache-ignite-users.70518.x6.nabble.com/Disable-Write
>>> Behind-tp11729p11984.html
>>> To unsubscribe from Disable WriteBehind, click here.
>>> NAML
>>> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>
>>
>> *ignite-3131d2bb.0.log* (7K) Download Attachment
>> <http://apache-ignite-users.70518.x6.nabble.com/attachment/11987/0/ignite-3131d2bb.0.log>
>>
>> ------------------------------
>> View this message in context: Re: Disable WriteBehind
>> <http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p11987.html>
>> Sent from the Apache Ignite Users mailing list archive
>> <http://apache-ignite-users.70518.x6.nabble.com/> at Nabble.com.
>>
>
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-ignite-users.70518.x6.nabble.com/Disable-
> WriteBehind-tp11729p11993.html
> To unsubscribe from Disable WriteBehind, click here
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=11729&code=amVzc2llLmppYW53ZWkubGluQGdtYWlsLmNvbXwxMTcyOXwtOTM2MTAxMjg=>
> .
> NAML
> <http://apache-ignite-users.70518.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>


ignite-8eb548de.0.log (8K) <http://apache-ignite-users.70518.x6.nabble.com/attachment/12050/0/ignite-8eb548de.0.log>




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Disable-WriteBehind-tp11729p12050.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Mime
View raw message