flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From an0 <an0...@gmail.com>
Subject Re: I want to use MapState on an unkeyed stream
Date Wed, 08 May 2019 18:44:39 GMT
I switched to using operator list state. It is more clear. It is also supported by RocksDBKeyedStateBackend,
isn't it?

On 2019/05/08 14:42:36, Till Rohrmann <trohrmann@apache.org> wrote: 
> Hi,
> 
> if you want to increase the parallelism you could also pick a key randomly
> from a set of keys. The price you would pay is a shuffle operation (network
> I/O) which would not be needed if you were using the unkeyed stream and
> used the operator list state.
> 
> However, with keyed state you could also use Flink's
> RocksDBKeyedStateBackend which allows to go out of core if your state size
> should grow very large.
> 
> Cheers,
> Till
> 
> On Tue, May 7, 2019 at 5:57 PM an0 <an00na@gmail.com> wrote:
> 
> > But I only have one stream, nothing to connect it to.
> >
> > On 2019/05/07 00:15:59, Averell <lvhuyen@gmail.com> wrote:
> > > From my understanding, having a fake keyBy (stream.keyBy(r =>
> > "dummyString"))
> > > means there would be only one slot handling the data.
> > > Would a broadcast function [1] work for your case?
> > >
> > > Regards,
> > > Averell
> > >
> > > [1]
> > >
> > https://ci.apache.org/projects/flink/flink-docs-stable/dev/stream/state/broadcast_state.html
> > >
> > >
> > >
> > > --
> > > Sent from:
> > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
> > >
> >
> 

Mime
View raw message