flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Import Configuration File in Flink Cluster
Date Mon, 23 May 2016 15:52:19 GMT
Hi Simon,

AFAIK this is the way to go. We could add a method to the
ParameterTool which loads from a resource to make it more convenient.

Cheers,
Max

On Mon, May 23, 2016 at 4:42 PM, simon peyer <simon.peyer@soom-it.ch> wrote:
> Hi
>
> @Max
> So for each file in the src/main/resources folder, I first have to create a
> new file, copy the file from the resources folder to this new file and then
> I'm able to parse it?
>
> @Stefano
> I think the files in src/main/resources  are integrated automatically right?
> Or am I missing something?
>
> Cheers
> Simon
>
>
> On 23 May 2016, at 16:30, Stefano Baghino <stefano.baghino@radicalbit.io>
> wrote:
>
> Are you using Maven to package your project? I believe the resources
> plugin[1] can suit your needs.
>
> [1]:
> http://maven.apache.org/plugins/maven-resources-plugin/examples/include-exclude.html
>
>
> On 23 May 2016, at 16:28, Maximilian Michels <mxm@apache.org> wrote:
>
> Hi Simon,
>
> You'll have to write the property file to disk first to load it using
> the ParameterTool.fromPropertiesFile method.
>
> For example:
>
> // copy config from Java resource to a file
> File configOnDisk = new File("/path/to/config.properties");
> Files.copy(getClass.getClassLoader.getResourceAsStream("config.properties"),
> configOnDisk.toPath());
> // load the new file
> ParameterTool.fromPropertiesFile(configOnDisk);
>
>
> Cheers,
> Max
>
>
>
> On Mon, May 23, 2016 at 3:56 PM, simon peyer <simon.peyer@soom-it.ch> wrote:
>>
>> Hi together
>>
>> Currently I'm using flink on a docker cluster on AWS.
>> I would like to use property files with the integrated
>> ParameterTool.fromPropertiesFile function of Flink.
>>
>> Locally this version works absolutely fine:
>> val configuration =
>> ParameterTool.fromPropertiesFile("src/main/resources/config.properties")
>>
>> But on the cluster this didn't work, so we introduced this version, which
>> also doesn't work:
>>
>> val baseParameters =
>> ParameterTool.fromPropertiesFile(getClass.getClassLoader.getResource("config.properties").getFile)
>>
>> gives
>>
>> java.io.FileNotFoundException: Properties file
>> file:/tmp/flink-web-upload-57bcc912-bc98-4c89-b5ee-c5176155abd5/992186c1-b3c3-4342-a5c8-67af133155e4pipeline-0.1.0-all.jar!/config.properties
>> does not exist
>>
>> The property file is located in src/main/resources.
>> Do you have any idea of how to integrate such property files into the jar
>> package?
>>
>> -Thanks
>> Simon
>
>
>
>
> --
> BR,
> Stefano Baghino
>
> Software Engineer @ Radicalbit
>
>

Mime
View raw message