Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 7C1CC2009F9 for ; Mon, 23 May 2016 16:42:37 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 7AEA0160A0E; Mon, 23 May 2016 14:42:37 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 75034160A05 for ; Mon, 23 May 2016 16:42:36 +0200 (CEST) Received: (qmail 12059 invoked by uid 500); 23 May 2016 14:42:35 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 12049 invoked by uid 99); 23 May 2016 14:42:35 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 May 2016 14:42:35 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 257B9CB5B8 for ; Mon, 23 May 2016 14:42:35 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.279 X-Spam-Level: * X-Spam-Status: No, score=1.279 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=soom-it-ch.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id F4t3FuX59phB for ; Mon, 23 May 2016 14:42:33 +0000 (UTC) Received: from mail-wm0-f42.google.com (mail-wm0-f42.google.com [74.125.82.42]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 8A9145F244 for ; Mon, 23 May 2016 14:42:32 +0000 (UTC) Received: by mail-wm0-f42.google.com with SMTP id a136so26437714wme.0 for ; Mon, 23 May 2016 07:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soom-it-ch.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:references:to:in-reply-to; bh=rS8YwRLiJjMUvmAB/86i541cIRe9gEvOZJ46uKu6Kd8=; b=gQXw9sVAqeQS8UA2V/mWugo/TE5Ddr9Z2cusxO8F5XUCDxYD2Lofqqutz+IqWa1SdY 8XJFS0FB3FZMT3+arysId+TENh2vu6GjppCrhgHWvnIZOCMSVRHFDrK3ADuC7ZBrbSSB bR0FzzGPFeU2m3YiWlSRBLUy18Xu/0xEJ2n8PzMCBjQ3XRLNhs5lqIq7Ou3sc/HmoXBS sAb8IICRLS2p5Z4sRUh+Pmxpvz2QQUFK8Vk7szxOaP5jIMCV41J3dkwfoKjry3KMggKP C8cQbnY3cma61mecQ0dfbi3eJngdLheQHAPGW+cKUBRTmccVtNVxCbF3gHeLmi82QR4M 8VXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:message-id:mime-version:subject:date :references:to:in-reply-to; bh=rS8YwRLiJjMUvmAB/86i541cIRe9gEvOZJ46uKu6Kd8=; b=mAVbs0ALdWqOqo/+cfXJG7jm6ZFQkdK+VB/WUjIQreaCwB0IQxf8VStXZa3OPFhiuH +z8rCJX3xQWIezuL3Ses0j3nlzYjz9gJEfP4laX2LLnbDoCNafx6EXNwt23jRWLLPUhq cvSPb+Rfa9FrHCKznEI+Y5wqda0Ezyc5AxTnKp9Q06/lomOXMxSC+Y2fQ6ekR+8spuFT BfydFqmBIXd4W9XP2FAoZ84HxxQHpRqikKpFQWqkGAjZVFCzTDzNoMHMS0+k7Zii78q0 HskIEUcNYIFpnjMtV1RMkO6fuz5CEfQNsOg3zAtIqmjf7ZZKeubwrXisXap9uuvNZXgk CHnA== X-Gm-Message-State: ALyK8tJbyQ/usqsmqHjHcu87n6ITLt+mMdDifYzJYdnWJAi8RWBZrwykKFnrLwxnyH+Q2MC2 X-Received: by 10.28.94.194 with SMTP id s185mr5670320wmb.62.1464014550696; Mon, 23 May 2016 07:42:30 -0700 (PDT) Received: from [192.168.0.57] (164.10.24.31.ftth.as8758.net. [31.24.10.164]) by smtp.gmail.com with ESMTPSA id b207sm14158685wmb.0.2016.05.23.07.42.28 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 23 May 2016 07:42:28 -0700 (PDT) From: simon peyer Content-Type: multipart/alternative; boundary="Apple-Mail=_82EDE5F7-60DB-4553-B66E-E0880C4029F0" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Import Configuration File in Flink Cluster Date: Mon, 23 May 2016 16:42:28 +0200 References: <9953CDAB-0EDD-46CA-9201-0F5C3DE0217A@soom-it.ch> To: user@flink.apache.org In-Reply-To: X-Mailer: Apple Mail (2.3112) archived-at: Mon, 23 May 2016 14:42:37 -0000 --Apple-Mail=_82EDE5F7-60DB-4553-B66E-E0880C4029F0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi=20 @Max So for each file in the src/main/resources folder, I first have to = create a new file, copy the file from the resources folder to this new = file and then I'm able to parse it? @Stefano I think the files in src/main/resources are integrated automatically = right? Or am I missing something? Cheers Simon > On 23 May 2016, at 16:30, Stefano Baghino = wrote: >=20 > Are you using Maven to package your project? I believe the resources = plugin[1] can suit your needs. >=20 > [1]: = http://maven.apache.org/plugins/maven-resources-plugin/examples/include-ex= clude.html = On 23 May 2016, at 16:28, Maximilian Michels wrote: Hi Simon, You'll have to write the property file to disk first to load it using the ParameterTool.fromPropertiesFile method. For example: // copy config from Java resource to a file File configOnDisk =3D new File("/path/to/config.properties"); = Files.copy(getClass.getClassLoader.getResourceAsStream("config.properties"= ), configOnDisk.toPath()); // load the new file ParameterTool.fromPropertiesFile(configOnDisk); Cheers, Max >=20 > On Mon, May 23, 2016 at 3:56 PM, simon peyer > wrote: > Hi together >=20 > Currently I'm using flink on a docker cluster on AWS. > I would like to use property files with the integrated = ParameterTool.fromPropertiesFile function of Flink. >=20 > Locally this version works absolutely fine: > val configuration =3D = ParameterTool.fromPropertiesFile("src/main/resources/config.properties") >=20 > But on the cluster this didn't work, so we introduced this version, = which also doesn't work: >=20 > val baseParameters =3D = ParameterTool.fromPropertiesFile(getClass.getClassLoader.getResource("conf= ig.properties").getFile) >=20 > gives=20 > java.io.FileNotFoundException: Properties file = file:/tmp/flink-web-upload-57bcc912-bc98-4c89-b5ee-c5176155abd5/992186c1-b= 3c3-4342-a5c8-67af133155e4pipeline-0.1.0-all.jar!/config.properties does = not exist > The property file is located in src/main/resources. > Do you have any idea of how to integrate such property files into the = jar package? >=20 > -Thanks > Simon >=20 >=20 >=20 > --=20 > BR, > Stefano Baghino >=20 > Software Engineer @ Radicalbit --Apple-Mail=_82EDE5F7-60DB-4553-B66E-E0880C4029F0 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii
Hi 

@Max
So for each file in the src/main/resources folder, I first = have to create a new file, copy the file from the resources folder to = this new file and then I'm able to parse it?

@Stefano
I think the files in src/main/resources  are integrated = automatically right? Or am I missing something?

Cheers
Simon


On = 23 May 2016, at 16:30, Stefano Baghino <stefano.baghino@radicalbit.io> wrote:

Are you using Maven to package your project? I believe the = resources plugin[1] can suit your needs.

[1]: http://maven.apache.org/plugins/maven-resources-plugin/examples= /include-exclude.html

On 23 May 2016, at 16:28, = Maximilian Michels <mxm@apache.org> wrote:

Hi Simon,

You'll have to write the property file to disk = first to load it using
the = ParameterTool.fromPropertiesFile method.

For = example:

// copy config from Java resource = to a file
File configOnDisk =3D new = File("/path/to/config.properties");
Files.copy(getClass.getClassLoader.getResourceAsStream("config.= properties"),
configOnDisk.toPath());
// = load the new file
ParameterTool.fromPropertiesFile(configOnDisk);


Cheers,
Max



On Mon, May 23, 2016 at 3:56 PM, = simon peyer <simon.peyer@soom-it.ch> wrote:
Hi together

Currently I'm using flink on a docker cluster on = AWS.
I would like to use property files with the = integrated ParameterTool.fromPropertiesFile function of Flink.

Locally this version = works absolutely fine:
val configuration =3D = ParameterTool.fromPropertiesFile("src/main/resources/config.propert= ies")

But on the cluster this didn't work, so = we introduced this version, which also doesn't work:

val = baseParameters = =3D = ParameterTool.fromPropertiesFile(getClass.getClassLoader.getResource("config.properties").getFile)

gives 
java.io.FileNotFoundException: Properties =
file =
file:/tmp/flink-web-upload-57bcc912-bc98-4c89-b5ee-c5176155abd5/992186c1-b=
3c3-4342-a5c8-67af133155e4pipeline-0.1.0-all.jar!/config.properties does =
not exist
The property file is located in = src/main/resources.
Do you have any idea of = how to integrate such property files into the jar package?

-Thanks
Simon



--
BR,
Stefano Baghino

Software Engineer @ Radicalbit
=

= --Apple-Mail=_82EDE5F7-60DB-4553-B66E-E0880C4029F0--