From user-return-22446-archive-asf-public=cust-asf.ponee.io@flink.apache.org Tue Aug 28 14:59:07 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 6DFA0180621 for ; Tue, 28 Aug 2018 14:59:06 +0200 (CEST) Received: (qmail 41753 invoked by uid 500); 28 Aug 2018 12:59:05 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 41743 invoked by uid 99); 28 Aug 2018 12:59:05 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Aug 2018 12:59:05 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id C53F6C08FE for ; Tue, 28 Aug 2018 12:59:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.189 X-Spam-Level: * X-Spam-Status: No, score=1.189 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=zendesk.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id TFH5DqzBOOZe for ; Tue, 28 Aug 2018 12:59:01 +0000 (UTC) Received: from us-smtp-delivery-110.mimecast.com (us-smtp-delivery-110.mimecast.com [216.205.24.110]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id D72C85F39F for ; Tue, 28 Aug 2018 12:59:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zendesk.com; s=mimecast20150210; t=1535461134; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nLtiBjAH/JdoEP1h0tBKNDXfw6VroAoCyhu/2Hnc75E=; b=SUvmkLNklCfAuqUk1++LRHDc+XM78Ps1onSQ6Mz44TNDV9BHm2GmLJsF5fC5y1mjdVhbAUudvJMxsGt0TZjNBs5idJfpXaKFeIwtkGSPlvo6L/3nwKXMnBmdKmlHJ9EVv+5hB2wfXg1niBf8xD4rOOJ9XAC6v45WC+I1XmquanM= Received: from mail-oi0-f69.google.com (mail-oi0-f69.google.com [209.85.218.69]) (Using TLS) by us-smtp-1.mimecast.com with ESMTP id us-mta-291-z3-Rt1ZTNTaa7m1T5S7hSA-1; Tue, 28 Aug 2018 08:58:49 -0400 Received: by mail-oi0-f69.google.com with SMTP id r131-v6so1122123oie.14 for ; Tue, 28 Aug 2018 05:58:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cxVWinUtVQUKznUN2smUHM5q41Ll8DoDIsyOGSZtUNc=; b=Z6NEzhRyDZJHA0gXfGFEAfpZS7Rn/7WayUvplFrhvxfZVQicEKcKV0lHVAnOb4fASO mKrt5dW5Cw5PE2gzU7S0F40vQ/Yqe1kbSrtq5jGpP8lhM2Xsc+CYWM5VAjAjB99DR1GR KWgGWH821KRdOW1M5e9xQxrO9sLoq1qPc3aa7Kvs/gs6ciz4IsqFRY+vIPzA5THvDQTe v9YYp4rlfYQjvQ1l+3tuUk0nJMueZSLWE+/DSQAFjWigq/waPZktgGd9P7dOI2Kjnd5O zHhtEuZ4KMhbsPfgc7JbDtTXZ4B8RPimG2Hy+QgVzCWDwXbQwPil/UIiTaHYkxXpAyej 7e9w== X-Gm-Message-State: APzg51AaObRyJeNQpWH90gOE7GdjMOpjaqdanFcUvAHwC2XpwPAJY0nT ucpydNdHv0yNcd0rP6/nxLO9Hfsg4mYaZgBHK7DqC1GilJ9OdxKvrOa0SsucE8GjeVrzV4ZJkmG 56eZVcAg4BLtC3cHy+JnoZjDFlPz2VfF+BVt0DL5bAR6O X-Received: by 2002:aca:4aca:: with SMTP id x193-v6mr1287676oia.211.1535461128779; Tue, 28 Aug 2018 05:58:48 -0700 (PDT) X-Google-Smtp-Source: ANB0VdawhoxgP2hW11u3zwOOJ1gXv2hEeSM09Q2tMBjWXqiaRs6XJAU64ZTPoaTd0vRdHiMNcUR5/5BMoeD6YKw3wG8= X-Received: by 2002:aca:4aca:: with SMTP id x193-v6mr1287657oia.211.1535461128363; Tue, 28 Aug 2018 05:58:48 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Hao Sun Date: Tue, 28 Aug 2018 05:58:37 -0700 Message-ID: Subject: Re: anybody can start flink with job mode? To: Till Rohrmann Cc: Chesnay Schepler , user , vino yang X-MC-Unique: z3-Rt1ZTNTaa7m1T5S7hSA-1 Content-Type: multipart/alternative; boundary="0000000000001f931a05747e676c" --0000000000001f931a05747e676c Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks Till for the follow up, I can run my job now. On Tue, Aug 28, 2018, 00:57 Till Rohrmann wrote: > Hi Hao, > > Vino is right, you need to specify the -j/--job-classname option which > specifies the job name you want to execute. Please make sure that the jar > containing this class is on the class path. > > I recently pushed some fixes which generate a better error message than > the one you've received. If you check out the latest master branch, then = it > should work better. > > Let me know if you should run into other problems. > > Cheers, > Till > > On Sat, Aug 25, 2018 at 5:11 AM Hao Sun wrote: > >> Thanks, I'll look into it. >> >> On Fri, Aug 24, 2018, 19:44 vino yang wrote: >> >>> Hi Hao Sun, >>> >>> From the error log, it seems that the jar package for the job was not >>> found. >>> You must make sure your Jar is in the classpath. >>> Related documentation may not be up-to-date, and there is a discussion >>> on this issue on this mailing list. [1] >>> >>> I see that the status of FLINK-10001 [2] is closed and it will be >>> updated with the release of 1.6.1 and 1.7.0. >>> >>> [1]: >>> http://mail-archives.apache.org/mod_mbox/flink-dev/201808.mbox/%3CCAC27= z=3DOaohMbmcryB-+m3GBmZP=3DXPHa8miHv7ZS1gRGSEkkfdg@mail.gmail.com%3E >>> >>> [2]: https://issues.apache.org/jira/browse/FLINK-10001 >>> >>> >>> Thanks, vino. >>> >>> >>> Hao Sun =E4=BA=8E2018=E5=B9=B48=E6=9C=8825=E6=97=A5= =E5=91=A8=E5=85=AD =E4=B8=8A=E5=8D=886:37=E5=86=99=E9=81=93=EF=BC=9A >>> >>>> I got an error like this. >>>> >>>> $ docker run -it flink-job:latest job-cluster >>>> Starting the job-cluster >>>> config file: >>>> jobmanager.rpc.address: localhost >>>> jobmanager.rpc.port: 6123 >>>> jobmanager.heap.size: 1024m >>>> taskmanager.heap.size: 1024m >>>> taskmanager.numberOfTaskSlots: 1 >>>> parallelism.default: 1 >>>> rest.port: 8081 >>>> Starting standalonejob as a console application on host cf9bd047082c. >>>> 2018-08-24 22:33:00,773 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>> ----------------------------------------------------------------------= ---------- >>>> 2018-08-24 22:33:00,774 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Starting >>>> StandaloneJobClusterEntryPoint (Version: 1.6.0, Rev:ff472b4, >>>> Date:07.08.2018 @ 13:31:13 UTC) >>>> 2018-08-24 22:33:00,775 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - OS current use= r: >>>> flink >>>> 2018-08-24 22:33:01,168 WARN org.apache.hadoop.util.NativeCodeLoader - >>>> Unable to load native-hadoop library for your platform... using >>>> builtin-java classes where applicable >>>> 2018-08-24 22:33:01,232 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Current >>>> Hadoop/Kerberos user: flink >>>> 2018-08-24 22:33:01,232 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM: OpenJDK 6= 4-Bit >>>> Server VM - Oracle Corporation - 1.8/25.111-b14 >>>> 2018-08-24 22:33:01,232 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Maximum heap s= ize: >>>> 981 MiBytes >>>> 2018-08-24 22:33:01,232 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JAVA_HOME: >>>> /usr/lib/jvm/java-1.8-openjdk/jre >>>> 2018-08-24 22:33:01,236 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Hadoop version= : >>>> 2.8.3 >>>> 2018-08-24 22:33:01,236 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Options: >>>> 2018-08-24 22:33:01,236 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xms1024m >>>> 2018-08-24 22:33:01,236 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx1024m >>>> 2018-08-24 22:33:01,237 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>> -Dlog4j.configuration=3Dfile:/opt/flink-1.6.0/conf/log4j-console.prope= rties >>>> 2018-08-24 22:33:01,237 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>> -Dlogback.configurationFile=3Dfile:/opt/flink-1.6.0/conf/logback-conso= le.xml >>>> 2018-08-24 22:33:01,237 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Program Argume= nts: >>>> 2018-08-24 22:33:01,237 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - --configDir >>>> 2018-08-24 22:33:01,238 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>> /opt/flink-1.6.0/conf >>>> 2018-08-24 22:33:01,238 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath: >>>> /opt/flink-1.6.0/lib/flink-python_2.11-1.6.0.jar:/opt/flink-1.6.0/lib/= flink-shaded-hadoop2-uber-1.6.0.jar:/opt/flink-1.6.0/lib/job.jar:/opt/flink= -1.6.0/lib/log4j-1.2.17.jar:/opt/flink-1.6.0/lib/slf4j-log4j12-1.7.7.jar:/o= pt/flink-1.6.0/lib/flink-dist_2.11-1.6.0.jar::: >>>> 2018-08-24 22:33:01,238 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - >>>> ----------------------------------------------------------------------= ---------- >>>> 2018-08-24 22:33:01,240 INFO >>>> org.apache.flink.runtime.entrypoint.ClusterEntrypoint - Registered UNI= X >>>> signal handlers for [TERM, HUP, INT] >>>> 2018-08-24 22:33:01,248 ERROR *org.apache.flink.runtime.entrypoint.Clu= sterEntrypoint >>>> - Could not parse command line arguments [--configDir, >>>> /opt/flink-1.6.0/conf].* >>>> org.apache.flink.runtime.entrypoint.FlinkParseException: Failed to >>>> parse the command line arguments. >>>> at >>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(Com= mandLineParser.java:52) >>>> at >>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.m= ain(StandaloneJobClusterEntryPoint.java:143) >>>> Caused by: org.apache.commons.cli.MissingOptionException: *Missing >>>> required option: j* >>>> at >>>> org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParse= r.java:199) >>>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:130) >>>> at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java:81) >>>> at >>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.parse(Com= mandLineParser.java:50) >>>> ... 1 more >>>> Exception in thread "main" java.lang.IllegalArgumentException: >>>> cmdLineSyntax not provided >>>> at >>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:546) >>>> at >>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:492) >>>> at >>>> org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:408) >>>> at >>>> org.apache.flink.runtime.entrypoint.parser.CommandLineParser.printHelp= (CommandLineParser.java:60) >>>> at >>>> org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.m= ain(StandaloneJobClusterEntryPoint.java:146) >>>> >>> --0000000000001f931a05747e676c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks Till for the follow up, I can run my job now.

On Tue, Aug 28, 2018, 00:57 Till Rohrmann <= trohrmann@apache.org> wrote:=
Hi Hao,

<= /div>
Vino is right, you need to specify the -j/--job-classname option = which specifies the job name you want to execute. Please make sure that the= jar containing this class is on the class path.=C2=A0

=
I recently pushed some fixes which generate a better error message tha= n the one you've received. If you check out the latest master branch, then = it should work better.

Let me know if you should r= un into other problems.

Cheers,
Till

On Sat, Aug 25, 201= 8 at 5:11 AM Hao Sun <hasun@zendesk.com> wrote:
Thanks, I'll look into it.

On Fri, Aug 24, 2018= , 19:44 vino yang <yanghua1127@gmail.com> wrote:
Hi Hao Sun,

From the error log,= it seems that the jar package for the job was not found.=C2=A0
Y= ou must make sure your Jar is in the classpath.=C2=A0
Related doc= umentation may not be up-to-date, and there is a discussion on this issue o= n this mailing list. [1]

I see that the status= of FLINK-10001 [2] is closed and it will be updated with the release of 1.= 6.1 and 1.7.0.

[2]:=C2=A0https://issues.apache.org/jira/browse/FLINK-10001

Thanks, vino.


Hao Sun <hasun@zendesk.com> =E4=BA=8E2018=E5=B9=B4= 8=E6=9C=8825=E6=97=A5=E5=91=A8=E5=85=AD =E4=B8=8A=E5=8D=886:37=E5=86=99=E9= =81=93=EF=BC=9A
I got an error like this.
$ docker run -it flink-job:latest job-cluster
Starting the job-cl= uster
config file:
jobmanager.rpc.address: localhost
jobmanager.rp= c.port: 6123
jobmanager.heap.size: 1024m
taskmanager.heap.size: 1024m=
taskmanager.numberOfTaskSlots: 1
parallelism.default: 1
rest.port= : 8081
Starting standalonejob as a console application on host cf9bd0470= 82c.
2018-08-24 22:33:00,773 INFO org.apache.flink.runtime.entrypoint.C= lusterEntrypoint - ------------------------------------------------= --------------------------------
2018-08-24 22:33:00,774 INFO org.apach= e.flink.runtime.entrypoint.ClusterEntrypoint - Starting Standalone= JobClusterEntryPoint (Version: 1.6.0, Rev:ff472b4, Date:07.08.2018 @ 13:31:= 13 UTC)
2018-08-24 22:33:00,775 INFO org.apache.flink.runtime.entrypoin= t.ClusterEntrypoint - OS current user: flink
2018-08-24 22:33:0= 1,168 WARN org.apache.hadoop.util.NativeCodeLoader -= Unable to load native-hadoop library for your platform... using builtin-ja= va classes where applicable
2018-08-24 22:33:01,232 INFO org.apache.fli= nk.runtime.entrypoint.ClusterEntrypoint - Current Hadoop/Kerberos = user: flink
2018-08-24 22:33:01,232 INFO org.apache.flink.runtime.entry= point.ClusterEntrypoint - JVM: OpenJDK 64-Bit Server VM - Oracle C= orporation - 1.8/25.111-b14
2018-08-24 22:33:01,232 INFO org.apache.fli= nk.runtime.entrypoint.ClusterEntrypoint - Maximum heap size: 981 M= iBytes
2018-08-24 22:33:01,232 INFO org.apache.flink.runtime.entrypoint= .ClusterEntrypoint - JAVA_HOME: /usr/lib/jvm/java-1.8-openjdk/jre<= br>2018-08-24 22:33:01,236 INFO org.apache.flink.runtime.entrypoint.Cluste= rEntrypoint - Hadoop version: 2.8.3
2018-08-24 22:33:01,236 INF= O org.apache.flink.runtime.entrypoint.ClusterEntrypoint - JVM Opt= ions:
2018-08-24 22:33:01,236 INFO org.apache.flink.runtime.entrypoint.= ClusterEntrypoint - -Xms1024m
2018-08-24 22:33:01,236 INFO = org.apache.flink.runtime.entrypoint.ClusterEntrypoint - -Xmx102= 4m
2018-08-24 22:33:01,237 INFO org.apache.flink.runtime.entrypoint.Clu= sterEntrypoint - -Dlog4j.configuration=3Dfile:/opt/flink-1.6.0/= conf/log4j-console.properties
2018-08-24 22:33:01,237 INFO org.apache.f= link.runtime.entrypoint.ClusterEntrypoint - -Dlogback.configura= tionFile=3Dfile:/opt/flink-1.6.0/conf/logback-console.xml
2018-08-24 22:= 33:01,237 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint = - Program Arguments:
2018-08-24 22:33:01,237 INFO org.apache.flink.= runtime.entrypoint.ClusterEntrypoint - --configDir
2018-08-2= 4 22:33:01,238 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint = - /opt/flink-1.6.0/conf
2018-08-24 22:33:01,238 INFO org.ap= ache.flink.runtime.entrypoint.ClusterEntrypoint - Classpath: /opt/= flink-1.6.0/lib/flink-python_2.11-1.6.0.jar:/opt/flink-1.6.0/lib/flink-shad= ed-hadoop2-uber-1.6.0.jar:/opt/flink-1.6.0/lib/job.jar:/opt/flink-1.6.0/lib= /log4j-1.2.17.jar:/opt/flink-1.6.0/lib/slf4j-log4j12-1.7.7.jar:/opt/flink-1= .6.0/lib/flink-dist_2.11-1.6.0.jar:::
2018-08-24 22:33:01,238 INFO org.= apache.flink.runtime.entrypoint.ClusterEntrypoint - ---------------= -----------------------------------------------------------------
2018-0= 8-24 22:33:01,240 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoi= nt - Registered UNIX signal handlers for [TERM, HUP, INT]
2018-0= 8-24 22:33:01,248 ERROR org.apache.flink.runtime.entrypoint.ClusterEntry= point - Could not parse command line arguments [--configDir, /opt/f= link-1.6.0/conf].
org.apache.flink.runtime.entrypoint.FlinkParseExce= ption: Failed to parse the command line arguments.
at org.apache.flink.= runtime.entrypoint.parser.CommandLineParser.parse(CommandLineParser.java:52= )
at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoi= nt.main(StandaloneJobClusterEntryPoint.java:143)
Caused by: org.apache.c= ommons.cli.MissingOptionException: Missing required option: j
at= org.apache.commons.cli.DefaultParser.checkRequiredOptions(DefaultParser.ja= va:199)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.jav= a:130)
at org.apache.commons.cli.DefaultParser.parse(DefaultParser.java= :81)
at org.apache.flink.runtime.entrypoint.parser.CommandLineParser.pa= rse(CommandLineParser.java:50)
... 1 more
Exception in thread "= main" java.lang.IllegalArgumentException: cmdLineSyntax not provided at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:546= )
at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.java:= 492)
at org.apache.commons.cli.HelpFormatter.printHelp(HelpFormatter.ja= va:408)
at org.apache.flink.runtime.entrypoint.parser.CommandLineParser= .printHelp(CommandLineParser.java:60)
at org.apache.flink.container.ent= rypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.= java:146)
--0000000000001f931a05747e676c--