spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michel Hubert <>
Subject updateStateByKey schedule time
Date Wed, 15 Jul 2015 12:42:03 GMT

I want to implement a time-out mechanism in de updateStateByKey(...) routine.

But is there a way the retrieve the time of the start of the batch corresponding to the call
to my updateStateByKey routines?

Suppose the streaming has build up some delay then a System.currentTimeMillis() will not be
the time of the time the batch was scheduled.

I want to retrieve the job/task schedule time of the batch for which my updateStateByKey(..)
routine is called.

Is this possible?

With kind regards,
Michel Hubert

View raw message