Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 0D6ED200B8C for ; Mon, 12 Sep 2016 08:41:29 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 0C02A160AC8; Mon, 12 Sep 2016 06:41:29 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 04A4A160AB8 for ; Mon, 12 Sep 2016 08:41:27 +0200 (CEST) Received: (qmail 26203 invoked by uid 500); 12 Sep 2016 06:41:27 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 26193 invoked by uid 99); 12 Sep 2016 06:41:27 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Sep 2016 06:41:27 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 9EFAE18014A for ; Mon, 12 Sep 2016 06:41:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.429 X-Spam-Level: * X-Spam-Status: No, score=1.429 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id dMwim5cCt-9z for ; Mon, 12 Sep 2016 06:41:24 +0000 (UTC) Received: from mail-lf0-f54.google.com (mail-lf0-f54.google.com [209.85.215.54]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id D45F25FAF1 for ; Mon, 12 Sep 2016 06:41:23 +0000 (UTC) Received: by mail-lf0-f54.google.com with SMTP id u14so80297888lfd.1 for ; Sun, 11 Sep 2016 23:41:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=yle9AAiypG3otqqT4TQXeT2AvNmiWsg59xFMe5NMiSQ=; b=x38jjWBEoPrIIGu2D2WIwYQB8SaCnLRPd/TAiniXNrBI9wEGAd0phl9nSiA2qZZSxa l6KAwL2Ylf2mNcSGwfYdI6SGQ+o/5Zhyug+Hop58WANdWLANd8SqD5vqNm0GyGsDIan+ yzclgiGTo55qLiK60vmtjT0Zp2WwCe7cxyiTGApg2S8wCarex9yfvrFHmrmWqVEC9oSq ldlu0b3qp0SXgwfEWf4TDPZC7/lTaEwIO/mZ+8BvsoBU22LEsl/b/bHhKtaqP6/ISLsK vWBK94wxW+KBls76HOUQXNISRLTTrREFh1zPc68Bzq6Y1vapHrbJmcA0l8izNQhUgNuQ bENg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=yle9AAiypG3otqqT4TQXeT2AvNmiWsg59xFMe5NMiSQ=; b=NdgzS12Ltu2cX1Jc4VhuJGXiJWG4h3+1u02VA5uymN5gbn03FWpGrCYq/CPw+eFE9k W+RqAoty8lVVVF1fCq0JyBfy+bmgGoJd5lkf8/1x+stk34+Q1h1KGP5wvWDgQyxH3B8Z Rkubh0KDI8HXmt70KsOMjppLhZwcUsKbiMvNB8eEF4ZJyQVMXIisbZFF/Nd/9NBqWpoK 62YhqjDnh8hyJMvGxcfQ5vq2aASN55qyryCQFpmpWrX+grwn7X2t7MPMpnktXxdBC9Jl 03DwaEzXhGZV0r+kI9e7jC12GDPGuCT+a0CN4tqR+GJhX3QDf2U/av3Fd3dEW6EMuT98 8EOA== X-Gm-Message-State: AE9vXwPeV/NGC/YZDaQR02tyHH3HOp4XdlQEoTZ0TeFedANRLDVJwblR7+oKp4DIR+8hGaAyD2l+7RaMnJbteA== X-Received: by 10.25.17.224 with SMTP id 93mr1500748lfr.77.1473662477160; Sun, 11 Sep 2016 23:41:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.114.83.10 with HTTP; Sun, 11 Sep 2016 23:41:16 -0700 (PDT) From: Swapnil Chougule Date: Mon, 12 Sep 2016 12:11:16 +0530 Message-ID: Subject: Flink JDBC JDBCOutputFormat Open To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a113fa7d87853e0053c49c846 archived-at: Mon, 12 Sep 2016 06:41:29 -0000 --001a113fa7d87853e0053c49c846 Content-Type: text/plain; charset=UTF-8 Hi Team, I want to know how tasknumber & numtasks help in opening db connection in Flink JDBC JDBCOutputFormat Open. I checked with docs where it says: taskNumber - The number of the parallel instance.numTasks - The number of parallel tasks.But couldn't get clear idea among parallel instance & parallel tasks. How do they contribute in concurrency with JDBC Source/Sink? I also checked with code but couldn't drill down further /** * Connects to the target database and initializes the prepared statement. * * @param taskNumber The number of the parallel instance. * @throws IOException Thrown, if the output could not be opened due to an * I/O problem. */ @Override public void open(int taskNumber, int numTasks) throws IOException { try { establishConnection(); upload = dbConn.prepareStatement(query); } catch (SQLException sqe) { throw new IllegalArgumentException("open() failed.", sqe); } catch (ClassNotFoundException cnfe) { throw new IllegalArgumentException("JDBC driver class not found.", cnfe); } } private void establishConnection() throws SQLException, ClassNotFoundException { Class.forName(drivername); if (username == null) { dbConn = DriverManager.getConnection(dbURL); } else { dbConn = DriverManager.getConnection(dbURL, username, password); } } Thanks, Swapnil --001a113fa7d87853e0053c49c846 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Team,

I want to know h= ow tasknumber & numtasks help in opening db connection in Flink JDBC JD= BCOutputFormat Open. I checked with docs where it says:
ta= skNumber - The number of the parallel instance.
numTas= ks - The number of parallel tasks.
But couldn't get cle= ar idea among parallel instance & parallel tasks. How do they contribut= e in concurrency with JDBC Source/Sink?

I also checked with co= de but couldn't drill down further

/*= * =20 =20
* Connects = to the target database and initializes the prepared statement.
*
* @param taskNumber The number of the parallel instance.
* @throws IOException Thrown, if the output could not be opened d= ue to an
* I/O problem.
*/
@Override
public void open(int taskNumber, int= numTasks) thr= ows IOException {
try {
establishConnect= ion();
upload =3D dbConn.prepa= reStatement(query);
} catch (SQLException = sqe) {
throw new IllegalArgumentException("open() failed.", sqe);
} catch (ClassNotFoundExcepti= on cnfe) {
throw new IllegalArgumentException("JDBC driver class not found.", cnfe);
}
}
private void establishConnection() th= rows SQLException, ClassNotFoundException {
Class.forName(driverna= me);
if (username =3D=3D null) {
dbConn =3D DriverManager.getConnection(dbURL);
} else {
dbConn =3D DriverManager.getConnection(dbURL, username, passwo= rd);
}
}

Thanks,
Swapnil
<= div>


--001a113fa7d87853e0053c49c846--