flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From simon peyer <simon.pe...@soom-it.ch>
Subject Re: Import Configuration File in Flink Cluster
Date Mon, 23 May 2016 14:42:28 GMT
Hi 

@Max
So for each file in the src/main/resources folder, I first have to create a new file, copy
the file from the resources folder to this new file and then I'm able to parse it?

@Stefano
I think the files in src/main/resources  are integrated automatically right? Or am I missing
something?

Cheers
Simon


> On 23 May 2016, at 16:30, Stefano Baghino <stefano.baghino@radicalbit.io> wrote:
> 
> Are you using Maven to package your project? I believe the resources plugin[1] can suit
your needs.
> 
> [1]: http://maven.apache.org/plugins/maven-resources-plugin/examples/include-exclude.html
<http://maven.apache.org/plugins/maven-resources-plugin/examples/include-exclude.html>

On 23 May 2016, at 16:28, Maximilian Michels <mxm@apache.org> wrote:

Hi Simon,

You'll have to write the property file to disk first to load it using
the ParameterTool.fromPropertiesFile method.

For example:

// copy config from Java resource to a file
File configOnDisk = new File("/path/to/config.properties");
Files.copy(getClass.getClassLoader.getResourceAsStream("config.properties"),
configOnDisk.toPath());
// load the new file
ParameterTool.fromPropertiesFile(configOnDisk);


Cheers,
Max


> 
> On Mon, May 23, 2016 at 3:56 PM, simon peyer <simon.peyer@soom-it.ch <mailto:simon.peyer@soom-it.ch>>
wrote:
> Hi together
> 
> Currently I'm using flink on a docker cluster on AWS.
> I would like to use property files with the integrated ParameterTool.fromPropertiesFile
function of Flink.
> 
> Locally this version works absolutely fine:
> val configuration = ParameterTool.fromPropertiesFile("src/main/resources/config.properties")
> 
> But on the cluster this didn't work, so we introduced this version, which also doesn't
work:
> 
> val baseParameters = ParameterTool.fromPropertiesFile(getClass.getClassLoader.getResource("config.properties").getFile)
> 
> gives 
> java.io.FileNotFoundException: Properties file file:/tmp/flink-web-upload-57bcc912-bc98-4c89-b5ee-c5176155abd5/992186c1-b3c3-4342-a5c8-67af133155e4pipeline-0.1.0-all.jar!/config.properties
does not exist
> The property file is located in src/main/resources.
> Do you have any idea of how to integrate such property files into the jar package?
> 
> -Thanks
> Simon
> 
> 
> 
> -- 
> BR,
> Stefano Baghino
> 
> Software Engineer @ Radicalbit


Mime
View raw message