beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sai Inampudi <sai.inamp...@gmail.com>
Subject Re: How to kick off a Beam pipeline with PortableRunner in Java?
Date Tue, 04 Dec 2018 22:21:00 GMT
Thanks for the help Ankur and Ruoyun, appreciate it. I went through the wiki and I am still
facing the same issue as before (where it complains about the following:
java.lang.IllegalArgumentException: Unknown 'runner' specified 'PortableRunner', supported
pipeline runners [DirectRunner, FlinkRunner, SparkRunner, TestFlinkRunner, TestSparkRunner])

I am probably doing a naive mistake but I am not sure where, below is everything I have done
thus far:
* Confirmed I have met the prerequisities in the wiki
     * Docker is installed and working without sudo
     * gradle target for the java container works

* Successfully started the JobServer (for now, I didn't bother with the flink job server and
instead just kicked off the reference job server) by running the following
    * ./gradlew :beam-runners-reference-job-server:run
    * Confirmed that the JobService started at 8099 port
        * Started ReferenceRunnerJobService at localhost:8099
           <============-> 98% EXECUTING [22m 43s]
           > :beam-runners-reference-job-server:run
           > IDLE

* My code itself is being managed via maven, so I made sure to pull in the latest 2.10.0-SNAPSHOT
dependencies as per https://repository.apache.org/content/repositories/snapshots/

* The pipeline is kicked off with the following options:
     * --runner=PortableRunner --jobEndpoint=localhost:8099 --streaming=true
     * Running the pipeline results in the following error
           * java.lang.IllegalArgumentException: Unknown 'runner' specified 'PortableRunner',
supported pipeline runners [DirectRunner, FlinkRunner, SparkRunner, TestFlinkRunner, TestSparkRunner]



I am sure I am missing something basic but I was hoping I could get ideas on what it might
be?


Mime
View raw message