flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Streaming statefull operator with hashmap
Date Wed, 11 Nov 2015 15:52:59 GMT
It should suffice to do something like

"getRuntimeContext().getKeyValueState("microModelMap", new
HashMap<InputType,MicroModel>().getClass(), null);"

Two more comments:

1) Making null the default value and initializing manually is probably more
efficient, because otherwise the empty map would have to be cloned each
time the default value is returned, which adds avoidable overhead.

2) The HashMap type will most likely go through Kryo, so for efficiency,
make sure you register the types "InputType" and "MicroModel" on the
execution environment.
    Here you need to do that manually, because they are type erased and
Flink cannot auto-register them.

Greetings,
Stephan



On Wed, Nov 11, 2015 at 4:32 PM, Gyula Fóra <gyula.fora@gmail.com> wrote:

> Hey,
>
> Yes what you wrote should work. You can alternatively use
> TypeExtractor.getForObject(modelMapInit) to extract the tye information.
>
> I also like to implement my custom type info for Hashmaps and the other
> types and use that.
>
> Cheers,
> Gyula
>
> Martin Neumann <mneumann@sics.se> ezt írta (időpont: 2015. nov. 11., Sze,
> 16:30):
>
> > Hej,
> >
> > What is the correct way of initializing a state-full operator that is
> using
> > a hashmap? modelMapInit.getClass() does not work neither does
> > HashMap.class. Do I have to implement my own TypeInformation class or is
> > there a simpler way?
> >
> > cheers Martin
> >
> > private OperatorState<HashMap<InputType,MicroModel>> microModelMap;
> >
> > @Override
> > public void open(Configuration parameters) throws Exception {
> >     HashMap<InputType,MicroModel> modelMapInit = new HashMap<>();
> >     this.microModelMap =
> > getRuntimeContext().getKeyValueState("microModelMap",
> > modelMapInit.getClass() , modelMapInit);
> > }
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message