flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaogang Shi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5917) Remove MapState.size()
Date Mon, 27 Feb 2017 01:49:46 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15885019#comment-15885019

Xiaogang Shi commented on FLINK-5917:

I think it's okay to remove this method. But a better choice is to use an in-memory cache
to record the size of the state. That way, we can achieve constant-time implementation of
the `size` method with little cost. I think this mechanism also works for the heap states
in the future.

> Remove MapState.size()
> ----------------------
>                 Key: FLINK-5917
>                 URL: https://issues.apache.org/jira/browse/FLINK-5917
>             Project: Flink
>          Issue Type: Improvement
>          Components: DataStream API
>    Affects Versions: 1.3.0
>            Reporter: Aljoscha Krettek
> I'm proposing to remove {{size()}} because it is a prohibitively expensive operation
and users might not be aware of it. Instead of {{size()}} users can use an iterator over all
mappings to determine the size, when doing this they will be aware of the fact that it is
a costly operation.
> Right now, {{size()}} is only costly on the RocksDB state backend but I think with future
developments on the in-memory state backend it might also become an expensive operation there.

This message was sent by Atlassian JIRA

View raw message