flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Bode <maximilian.b...@tngtech.com>
Subject Backpressure in the context of JDBCOutputFormat update
Date Thu, 21 Jan 2016 14:35:12 GMT
Hi everyone,

in a Flink (0.10.1) job with two JDBCOutputFormat sinks, one of them (doing a database update)
is performing slower than the other one (an insert). The job as a whole is also slow as upstream
operators are slowed down due to backpressure. I am able to speed up the whole job by introducing
an a priori unnecessary .distinct(), which of course blocks downstream execution of the slow
sink, which in turn seems to be able to execute faster when given all data at once.

Any ideas what is going on here? Is there something I can do without introducing unnecessary
computation steps?

Cheers,
Max
— 
Maximilian Bode * Junior Consultant * maximilian.bode@tngtech.com * 0176 1000 75 50
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
Sitz: Unterföhring * Amtsgericht München * HRB 135082


Mime
View raw message