tephra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andreas Neumann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEPHRA-247) Avoid encoding the transaction multiple times
Date Tue, 16 Jan 2018 17:47:00 GMT

    [ https://issues.apache.org/jira/browse/TEPHRA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16327452#comment-16327452

Andreas Neumann commented on TEPHRA-247:

Yes, the in-progress list can grow large. A would think it will rarely grow larger than a
few thousand, and it will always reflect the load on the system.

The invalid list, however, grows persistently and will not shrink when the load goes down.
We have seen some instances where an ill-behaved client could create 10s of 1000s of invalids
per day, and soon you have hundreds of thousands. These then impact the performance of the
system even when the load is low. That is why we are focussing more on the invalid list. 

Do you have ideas on how to reduce the in-progress list overhead?

> Avoid encoding the transaction multiple times
> ---------------------------------------------
>                 Key: TEPHRA-247
>                 URL: https://issues.apache.org/jira/browse/TEPHRA-247
>             Project: Tephra
>          Issue Type: Improvement
>          Components: core, manager
>    Affects Versions: 0.12.0-incubating
>            Reporter: Andreas Neumann
>            Assignee: Andreas Neumann
>            Priority: Major
>         Attachments: design.jpg
> Currently, the same transaction object is encoded again and again for every Get performed
in HBase. It would be better to cache the encoded transaction for the duration of the transaction
and reuse it, 

This message was sent by Atlassian JIRA

View raw message