Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
References: <CAHM4Zt=tMy1RAYt-Vfuoferqao7hAmTzqnY5DNOTh+zP-CLPJA@mail.gmail.com>
 <CAC27z=NANuCSHN_AivpCon-EGk2UqipBA_abdaXDzoBBmzxdRw@mail.gmail.com>
In-Reply-To: <CAC27z=NANuCSHN_AivpCon-EGk2UqipBA_abdaXDzoBBmzxdRw@mail.gmail.com>
From: Aljoscha Krettek <aljoscha@apache.org>
Date: Wed, 19 Oct 2016 09:28:31 +0000
Message-ID: <CANMXwW24Fo67q5BxFG64YrohkU8dmF3eyK3mC3r65=3y5sH53g@mail.gmail.com>
Subject: Re: Add MapState for keyed streams
To: dev@flink.apache.org
Content-Type: multipart/alternative; boundary=94eb2c1142d6562cf5053f346fa5
archived-at: Wed, 19 Oct 2016 09:28:45 -0000

--94eb2c1142d6562cf5053f346fa5
Content-Type: text/plain; charset=UTF-8

Hi,
just making sure I understand this correctly. Would the MapState keys be
the same keys as the one provided when creating the KeyedStream or a
different key.

As an example, would it be like this:

DataStream<Tuple2<K, V>> input = ...;
KeyedStream keyed = input.keyBy(0)
keyed.map( Tuple2<K, V> input -> mapState.put(input.f0, input.f1) )

or like this:

DataStream<Tuple3<K, V1, V2>> input = ...;
KeyedStream keyed = input.keyBy(0)
keyed.map( Tuple3<K, V1, V2> input -> mapState.put(input.f1, input.f2) ) //
<- notice how the key for the MapState is different from the key of the
KeyedStream

Cheers,
Aljoscha


On Wed, 19 Oct 2016 at 10:55 Till Rohrmann <trohrmann@apache.org> wrote:

Hi Xiaogang,

I really like your proposal and think that this would be a valuable
addition to Flink :-)

For convenience we could maybe add contains(K key), too.

Java's Map interface returns a Set of Entry when calling entrySet() (which
is the equivalent of iterator() in our interface). The Entry interface not
only allows to get access to the key and value of the map entry but also
allows to set a value for the respective key via setValue (even though it's
an optional operation). Maybe we want to offer something similar when
getting access to the entry set via the iterator method.

Cheers,
Till

On Wed, Oct 19, 2016 at 4:18 AM, SHI Xiaogang <shixiaogangg@gmail.com>
wrote:

> Hi, all. I created the JIRA https://issues.apache.org/
> jira/browse/FLINK-4856 to
> propose adding MapStates into Flink.
>
> MapStates are very useful in our daily jobs. For example, when
implementing
> DistinctCount, we store the values into a MapState and the result of each
> group(key) is exactly the number of entries in the MapState.
>
> In my opinion, the methods provided by the MapState may include:
> * void put(K key, V value)
> * V get(K key)
> * Iterable<K> keys()
> * Iterable<V> values()
> * Iterator<Tuple2<K, V>> iterator()
>
> Do you have any comments? Any is appreciated.
>
> Xiaogang
>

--94eb2c1142d6562cf5053f346fa5--