Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 33822200B35 for ; Tue, 5 Jul 2016 12:02:26 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 32342160A60; Tue, 5 Jul 2016 10:02:26 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 550DB160A4F for ; Tue, 5 Jul 2016 12:02:25 +0200 (CEST) Received: (qmail 75472 invoked by uid 500); 5 Jul 2016 10:02:24 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 75462 invoked by uid 99); 5 Jul 2016 10:02:24 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Jul 2016 10:02:24 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id F07D8C0F4A for ; Tue, 5 Jul 2016 10:02:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.43 X-Spam-Level: * X-Spam-Status: No, score=1.43 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id RagnGGnJN35H for ; Tue, 5 Jul 2016 10:02:21 +0000 (UTC) Received: from mail-it0-f42.google.com (mail-it0-f42.google.com [209.85.214.42]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 9468E5F1B3 for ; Tue, 5 Jul 2016 10:02:20 +0000 (UTC) Received: by mail-it0-f42.google.com with SMTP id g4so37283043ith.1 for ; Tue, 05 Jul 2016 03:02:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=UVuCLSD9HxzHPbXd27bBMg0Z3WVHuJIbm3s+j5Htvgw=; b=mq3cn4X/FJ8LVwvKqOV0TruzEw+D1S9OvTFcaQvZn61VKf4Xwk2997v8ymkF6gbm0t /Ci/480jbrwvrwsFENBQDQ8qVHICzdwmklE+eZYQGsyC9hC5jxDhQEVH0xMPr72AKxwu cd7UxypdlHdUC6aH/LZrYzdSUqDGomquAmVAiOOVKUovnX4eFfhgTn9cX4UQiq/zECgb TciCblQLhOhIHImD5dbZlz4r4PkhUet+2yn1utwJlHYZehjaZuy5Ab2du7tum73hC+I1 X99nbIqBIyv7cpAQ1iBhwizS3yehRYeClpeErAhW4SHRNgAVlRo+TaTxFOrGziJ8YrOb HLLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=UVuCLSD9HxzHPbXd27bBMg0Z3WVHuJIbm3s+j5Htvgw=; b=Tmc5tivfvG1fJPUF8oxBL2fbFPxMiUu6SDH8bWtSuqT0Gwyk6X8j+4DKlDWC/lTcEk PFaY6edIsUE9/3C+vWphV1f+h9/jmQHvg2vu6xVszToXhLfGQffokCXloULZ+IUhmP/j jNJlUI5GtAgp4wypLzG6qii/vHJSagLZ1rkIkOEowjzbJqPKZkV0orFyUiNip3KVCKu+ LaBMUNN9rDTklQkLUxN44adna0ZIkroVOV3OBh3+drgEzoaCCuctuixz8d233FE34PZ2 wqI2YFzcId3YgLOUT23ut492MPSKhDTq5YdkQQ5x6SJeItZ9YSI5HndAeaPVo0mOyy+b NqIA== X-Gm-Message-State: ALyK8tLuZbQZGFa9SJKlbYaHOX18yWo85/tvSI6csQQPQhNpcPX0bndzen0ylpIcUnuHTUjrzLd4SLZIETBAfA== X-Received: by 10.36.98.8 with SMTP id d8mr12178037itc.39.1467712939392; Tue, 05 Jul 2016 03:02:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.197.132 with HTTP; Tue, 5 Jul 2016 03:01:40 -0700 (PDT) In-Reply-To: References: From: Harikrishnan S Date: Tue, 5 Jul 2016 15:31:40 +0530 Message-ID: Subject: Re: JDBC sink in flink To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a113f75b26285cb0536e08cd6 archived-at: Tue, 05 Jul 2016 10:02:26 -0000 --001a113f75b26285cb0536e08cd6 Content-Type: text/plain; charset=UTF-8 Oh. So you mean if I write a custom sink for a db, I just need to create one connection in the open() method and then the invoke() method will reuse it ? Basically I need to do 35k-50k+ upserts in postgres. Can I reuse JDBCOutputFormat for this purpose ? I couldn't find a proper document describing how sinks works in flink. On Tue, Jul 5, 2016 at 2:41 PM, Stefano Bortoli wrote: > The connection will be managed by the splitManager, no need of using a > pool. However, if you had to, probably you should look into > establishConnection() method of the JDBCInputFormat. > > > > 2016-07-05 10:52 GMT+02:00 Flavio Pompermaier : > >> why do you need a connection pool? >> On 5 Jul 2016 11:41, "Harikrishnan S" wrote: >> >>> Hi, >>> >>> Are there any examples of implementing a jdbc sink in flink using a >>> connection pool ? >>> >>> Thanks >>> >>> On Tue, Jul 5, 2016 at 2:00 PM, Harikrishnan S >>> wrote: >>> >>>> Hi, >>>> >>>> Are there any examples of implementing a jdbc sink in flink using a >>>> connection pool ? >>>> >>>> Thanks >>>> >>>> On Tue, Jul 5, 2016 at 1:57 PM, Harikrishnan S >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> Are there any examples of implementing a jdbc sink in flink using a >>>>> connection pool ? >>>>> >>>>> Thanks >>>>> >>>> >>>> >>> > --001a113f75b26285cb0536e08cd6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Oh. So you mean if I write a custom sink for a db, I just need = to create one connection in the open() method and then the invoke() method = will reuse it ? Basically I need to do 35k-50k+ upserts in postgres. Can I = reuse=C2=A0JDBCOutputFormat for this= purpose ? I couldn't find a proper document describing how sinks works= in flink.=C2=A0



On Tue, Jul 5, 2016 at 2:41 PM, Stefano Bortoli <s.= bortoli@gmail.com> wrote:
<= div dir=3D"ltr">The connection will be managed by the splitManager, no need= of using a pool. However, if you had to, probably you should look into est= ablishConnection() method of the JDBCInputFormat.



2016-07-05 10:52 GMT+02:00 Flavio Pompermaier <pomperm= aier@okkam.it>:

why do you need a connection pool?

On 5 Jul 2016 11:41, "Harikrishnan S" = <hihari777@gmai= l.com> wrote:
Hi,

Are there any examples of= implementing a jdbc sink in flink using a connection pool ?

Thanks

On Tue, Jul 5, 2016 at 2:00 PM= , Harikrishnan S <hihari777@gmail.com> wrote:
Hi,

Are there any examples of implementing a jdbc sink in flink using a conn= ection pool ?
=
Thanks

On Tue, Jul 5, 2016 at 1:57 PM, Harikrishnan S <hihar= i777@gmail.com> wrote:
Hi,

Are there any examples of implementing a jdbc sink in flin= k using a connection pool ?

Thanks




--001a113f75b26285cb0536e08cd6--