tephra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Xiaoman Huang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEPHRA-246) Implement better transaction codecs
Date Tue, 27 Nov 2018 08:45:00 GMT

    [ https://issues.apache.org/jira/browse/TEPHRA-246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16700064#comment-16700064

Patrick Xiaoman Huang commented on TEPHRA-246:

Because inProgress array and invalidList array in a Transaction is sorted, that we can encoding
it with first value and rest by delta value.

But why would writePointer align with current time? this makes txid have 2 parts, one is time*MAX_TX_PER-MS
and other is a incremental value, I think this make encoding using delta value difficult.

If writePointer is just a incremental value, that means delta value should be very small,
I think just store delta values in variable bytes will reduce the size of inProgress and invalidList


> Implement better transaction codecs
> -----------------------------------
>                 Key: TEPHRA-246
>                 URL: https://issues.apache.org/jira/browse/TEPHRA-246
>             Project: Tephra
>          Issue Type: Improvement
>          Components: core, manager
>    Affects Versions: 0.12.0-incubating
>            Reporter: Andreas Neumann
>            Assignee: Andreas Neumann
>            Priority: Major
> The current codec simply uses Thrift. However, with large invalid lists, the encoded
byte array cvan become very large, Using compression techniques such as delta encoding, variable
length integer encoding, or simple compression can greatly reduce the size of the encoded
byte array, as well as the time it takes to encode, transmit, and decode the transaction.

> This can be applied to the transactions in the Thrift protocol as well as to the transactions
that are attached to the HBase operations. 

This message was sent by Atlassian JIRA

View raw message