flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prez Cannady <revp...@opencorrelate.org>
Subject Moving from single-node, Maven examples to cluster execution
Date Thu, 16 Jun 2016 21:51:18 GMT
Having a hard time trying to get my head around how to deploy my Flink programs to a pre-configured,
remote Flink cluster setup.

My Mavenized setup uses Spring Boot (to simplify class path handling and generate pretty logs)
to execute provision a StreamExecutionEnvironment with Kafka sources and sinks. I can also
run this quite effective the standard way (`java -jar …`).  What I’m unclear on is how
I might go about distributing this code to run on an existing Flink cluster setup.  Where
do I drop the jars? Do I need to restart Flink to do so?

class AppRunner extends CommandLineRunner {

    val log = LoggerFactory.getLogger(classOf[AppRunner])

    override def run(args: String*): Unit = {

        val env : StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment

    val consumer = …
    val producer = ...
        val stream = env.addSource(consumer)

        // Do some stuff



object App {

    def main( args: Array[String] ) : Unit = {
        SpringApplication.run(classOf[AppRunner], args: _*)

Try as I might, I couldn’t find any clear instructions on how to do this in the documentation.
 The cluster documentation ends with starting it.


The Wikiedits example doesn’t involve any third party dependencies, so I’m not clear on
how to manage class path for it.


Any help in getting me on the right, preferably best practices path would be appreciated.

Prez Cannady  
p: 617 500 3378  
e: revprez@opencorrelate.org <mailto:revprez@opencorrelate.org>  
GH: https://github.com/opencorrelate <https://github.com/opencorrelate>  
LI: https://www.linkedin.com/in/revprez <https://www.linkedin.com/in/revprez>  

View raw message