flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-12694) Implement a HeapAccountingManager
Date Wed, 01 Jul 2020 05:57:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-12694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149126#comment-17149126
] 

Yu Li commented on FLINK-12694:
-------------------------------

Thanks for the interest [~Anglenet], but this one is in-progress. Actually the whole spill-able
backend solution has already been uploaded onto [flink-packages|https://flink-packages.org/packages/spillable-state-backend-for-flink]
and we will merge the [codes|https://github.com/realtime-storage-engine/flink-spillable-statebackend]
to community gradually after careful review.

> Implement a HeapAccountingManager
> ---------------------------------
>
>                 Key: FLINK-12694
>                 URL: https://issues.apache.org/jira/browse/FLINK-12694
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / State Backends
>            Reporter: Yu Li
>            Assignee: Yu Li
>            Priority: Major
>
> We need a {{HeapAccountingManager}} for a) (roughly) estimating the on-heap size of each
key-value; and b) recording the size of each on-heap and off-heap KeyGroup. With such accountings
we could decide which KeyGroup to spill/load.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message