I have a stream application which is running out of heap space - looks like there might be something up with state storage.... I'm having trouble determining if it just needs a lot of memory or there is a memory leak.
Are there any tips/best practice for this? I've looked at heap dumps and I just see lot of tuples ... it's not clear which task/operator they belong to...
I'm looking at moving to use RockDB (assuming it's just memory ...) but I'd like to determine if there is actually a leak. It doesn't seem to have issues with smaller input rates.
For reference, I'm using 1.2.1 and the source is a 4 partition Kafka topic with some windowing etc after that...
Thanks for any pointers/advice.