Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <A13D0B42-9487-4EEB-AFBE-3B4B7E90F745@soom-it.ch>
References: <9953CDAB-0EDD-46CA-9201-0F5C3DE0217A@soom-it.ch>
 <CAK+0a_r3c0H2jeQ9AV=tBdKwvF+1Kpwer2EvLM3Y=6R41M5XMQ@mail.gmail.com> <A13D0B42-9487-4EEB-AFBE-3B4B7E90F745@soom-it.ch>
From: Maximilian Michels <mxm@apache.org>
Date: Mon, 23 May 2016 17:52:19 +0200
Message-ID: <CAGco--a6b5G4Z=PLneQwD=a7D4P8mqo=5iN_7Yn39DFv-3LeUw@mail.gmail.com>
Subject: Re: Import Configuration File in Flink Cluster
To: "user@flink.apache.org" <user@flink.apache.org>
Content-Type: text/plain; charset=UTF-8
archived-at: Mon, 23 May 2016 15:52:41 -0000

Hi Simon,

AFAIK this is the way to go. We could add a method to the
ParameterTool which loads from a resource to make it more convenient.

Cheers,
Max

On Mon, May 23, 2016 at 4:42 PM, simon peyer <simon.peyer@soom-it.ch> wrote:
> Hi
>
> @Max
> So for each file in the src/main/resources folder, I first have to create a
> new file, copy the file from the resources folder to this new file and then
> I'm able to parse it?
>
> @Stefano
> I think the files in src/main/resources  are integrated automatically right?
> Or am I missing something?
>
> Cheers
> Simon
>
>
> On 23 May 2016, at 16:30, Stefano Baghino <stefano.baghino@radicalbit.io>
> wrote:
>
> Are you using Maven to package your project? I believe the resources
> plugin[1] can suit your needs.
>
> [1]:
> http://maven.apache.org/plugins/maven-resources-plugin/examples/include-exclude.html
>
>
> On 23 May 2016, at 16:28, Maximilian Michels <mxm@apache.org> wrote:
>
> Hi Simon,
>
> You'll have to write the property file to disk first to load it using
> the ParameterTool.fromPropertiesFile method.
>
> For example:
>
> // copy config from Java resource to a file
> File configOnDisk = new File("/path/to/config.properties");
> Files.copy(getClass.getClassLoader.getResourceAsStream("config.properties"),
> configOnDisk.toPath());
> // load the new file
> ParameterTool.fromPropertiesFile(configOnDisk);
>
>
> Cheers,
> Max
>
>
>
> On Mon, May 23, 2016 at 3:56 PM, simon peyer <simon.peyer@soom-it.ch> wrote:
>>
>> Hi together
>>
>> Currently I'm using flink on a docker cluster on AWS.
>> I would like to use property files with the integrated
>> ParameterTool.fromPropertiesFile function of Flink.
>>
>> Locally this version works absolutely fine:
>> val configuration =
>> ParameterTool.fromPropertiesFile("src/main/resources/config.properties")
>>
>> But on the cluster this didn't work, so we introduced this version, which
>> also doesn't work:
>>
>> val baseParameters =
>> ParameterTool.fromPropertiesFile(getClass.getClassLoader.getResource("config.properties").getFile)
>>
>> gives
>>
>> java.io.FileNotFoundException: Properties file
>> file:/tmp/flink-web-upload-57bcc912-bc98-4c89-b5ee-c5176155abd5/992186c1-b3c3-4342-a5c8-67af133155e4pipeline-0.1.0-all.jar!/config.properties
>> does not exist
>>
>> The property file is located in src/main/resources.
>> Do you have any idea of how to integrate such property files into the jar
>> package?
>>
>> -Thanks
>> Simon
>
>
>
>
> --
> BR,
> Stefano Baghino
>
> Software Engineer @ Radicalbit
>
>