flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jingsong Li <jingsongl...@gmail.com>
Subject Re: SQL Performance
Date Wed, 27 Nov 2019 06:45:09 GMT
Hi Nicholas,

You can take a look to JDBCSinkFunction and JDBCUpsertSinkFunction.
They can set something like flush batch max size just like vino mentioned.
JDBCUpsertOutputFormat also has a async thread to flush batch to JDBC
 to avoid too high delay. In this way, the throughput can be improved while
 the delay is low.

Best,
Jingsong Lee

On Wed, Nov 27, 2019 at 10:16 AM vino yang <yanghua1127@gmail.com> wrote:

> Hi Nick,
>
> Can you provide more details? Are you using JDBCOutputFormat? If yes, can
> `JDBCOutputFormatBuilder#setBatchInterval` help you?
>
> Best,
> Vino
>
> Nicholas Walton <nwalton@me.com> 于2019年11月26日周二 下午9:20写道:
>
>> I’m streaming records down to an Embedded Derby database, at a rate of
>> around 200 records per second. I’m certain Derby can sustain a higher
>> throughput than that, if I could buffer the records but it seems that I’m
>> writing each record as soon as it arrives and as a single transaction which
>> is inefficient.
>>
>> Is there a way to to improve my throughput rate?
>>
>> Nick Walton
>
>

-- 
Best, Jingsong Lee

Mime
View raw message