Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
Message-Id: 
 <1460036466.4119473.571808673.019EFE9C@webmail.messagingengine.com>
From: Bart van Deenen <bartvandeenen@fastmail.fm>
To: user@flink.apache.org
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain
Subject: mutable hashmap outside of stream, does it get snapshotted ?
Date: Thu, 07 Apr 2016 15:41:06 +0200

Hi all

I'm having a datastream transformation, that updates a mutable 
hashmap that exists outside of the stream.

So it's something like

object FlinkJob {
  val uriLookup = mutable.HashMap.empty[String, Int]


  def main(args: Array[String]) {

    val stream: DataStream = ...

    stream.keybBy(1).timeWindow(..).fold(..)
    .window(..)
    .map(..).fold(..)
    .addSink(..)
  }
}

where the uriLookup hashmap gets updated inside the stream
transformation, 
and is serialized in the step before the addSink

It works fine, however

Does the snapshotting mechanism in case of a node failure actually
serialize this map?

And out of curiousity, can I actually see what data exists inside the
snapshot data?

Thanks.

Bart