zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From suyogmapara <...@git.apache.org>
Subject [GitHub] zookeeper issue #567: ZOOKEEPER-3071: Add a config parameter to control tran...
Date Thu, 06 Sep 2018 22:30:29 GMT
Github user suyogmapara commented on the issue:

    https://github.com/apache/zookeeper/pull/567
  
    
    Hi @maoling 
    
    I tested the patch with following parameters:
    
    1. PreAllocSize = 100 KB, TxnLogSizeLimit = 200 KB, 100 transactions of little over 10
KBs. Outcome: 5 transaction files with little over 200 KBs each (rounded up to last transaction).
    2. PreAllocSize = 100 KB, TxnLogSizeLimit = 100 KB, 100 transactions of little over 10
KBs. Outcome: 100 transaction files one per each transaction
    3. PreAllocSize = 100 KB, TxnLogSizeLimit = 50 KB, 100 transactions of little over 10
KBs. Outcome: 100 transaction files one per each transaction.
    
    Second and third outcomes are expected as every time it tries to pre-allocate more, we
would be rolling the log. Of course those won't be sensible configurations, but if someone
uses them, only performance would be impacted and not the correctness. Please let me know
if you disagree.
    
    @anmolnar 
    snapCount doesn't give direct handle on the transaction log size. Changing average transaction
size from 1 KB to 100 KB would increase the transaction log size by 2 orders of magnitude
and may push they sync time over the edge. More importantly reducing the snapCount would also
increase how frequently we snapshot and so we would be forced to find a sweet spot to balance
the two conflicting goals. On the other hand, txnLogSizeLimit provides cheap and independent
control on the worst case seek time during the diff sync.


---

Mime
View raw message