flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shaosu Liu <...@uber.com>
Subject Memory Management in Streaming?
Date Fri, 02 Sep 2016 20:45:55 GMT
Hi,

I have had issues when I processed large amount of data (large windows
where I could not do incremental updates), flink slowed down significantly.
It did help when I increased the amount of memory and used off heap
allocation. But it only delayed the onset of the probelm without solving
it.

Could some one give me some hints on how Flink manage window buffer and how
streaming manages its memory. I see this page on batch api memory
management and wonder what is the equivalent for streaming?
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741525

-- 
Cheers,
Shaosu

Mime
View raw message