Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: Use Single Sink For All windows
From: Aljoscha Krettek <aljoscha@apache.org>
In-Reply-To: <1747645.ifJNGfsv0h@nico-work>
Date: Mon, 12 Jun 2017 17:51:17 +0200
Cc: user <user@flink.apache.org>,
 rhashmi <rizhashmi@hotmail.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <B89672A0-DB8E-4A19-AD80-999AD1C0B6A4@apache.org>
References: <1496558868204-13475.post@n4.nabble.com>
 <1496758951159-13525.post@n4.nabble.com> <1747645.ifJNGfsv0h@nico-work>
To: Nico Kruber <nico@data-artisans.com>
archived-at: Mon, 12 Jun 2017 15:51:24 -0000

Ah, I think now I get your problem. You could manually implement =
batching inside your SinkFunction, The SinkFunction would batch values =
in memory and periodically (based on the count of values and on a =
timeout) send these values as a single batch to MySQL. To ensure that =
data is not lost you can implement the CheckpointedFunction interface =
and make sure to always flush to MySQL when a snapshot is happening.

Does that help?

> On 8. Jun 2017, at 11:46, Nico Kruber <nico@data-artisans.com> wrote:
>=20
> How about using asynchronous I/O operations?
>=20
> =
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/
> asyncio.html
>=20
>=20
> Nico
>=20
> On Tuesday, 6 June 2017 16:22:31 CEST rhashmi wrote:
>> because of parallelism i am seeing db contention. Wondering if i can =
merge
>> sink of multiple windows and insert in batch.
>>=20
>>=20
>>=20
>> --
>> View this message in context:
>> =
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-Si=
n
>> gle-Sink-For-All-windows-tp13475p13525.html Sent from the Apache =
Flink User
>> Mailing List archive. mailing list archive at Nabble.com.
>=20