tephra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andreas Neumann (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEPHRA-241) Introduce a way to limit the size of a transaction
Date Wed, 16 Aug 2017 18:52:00 GMT
Andreas Neumann created TEPHRA-241:

             Summary: Introduce a way to limit the size of a transaction
                 Key: TEPHRA-241
                 URL: https://issues.apache.org/jira/browse/TEPHRA-241
             Project: Tephra
          Issue Type: Improvement
          Components: api, manager
    Affects Versions: 0.12.0-incubating
            Reporter: Andreas Neumann
            Assignee: Andreas Neumann
             Fix For: 0.13.0-incubating

When clients perform a huge number of writes in a short transaction, that can result in huge
change sets. For example, if a client performs 10M writes and sends that change set over,
that can easily be 1GB large. The transaction manager will keep this in memory. It will also
write this as an edit to the transaction log.

Assume it runs out of memory because the change set is too large. It crashes and when it restarts,
it will replay the log, load that huge change set again, and crash again. 

To prevent this kind of systemic failure, and to encourage developers to use long transactions
when performing many writes, we can introduce two new properties in the configuration:

- change set warn threshold: if a change set exceeds this size, a warning is logged. 
- change set reject threshold: if a change set exceeds this size, it is rejected (canCommit
will throw an exception) and that will fail the transaction.

Both thresholds should be Long.MAX_VALUE by default, to preserve existing behavior after upgrade.

This message was sent by Atlassian JIRA

View raw message