flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zili Chen <wander4...@gmail.com>
Subject Re: Flink 1.8 run参数不一样
Date Wed, 24 Jul 2019 01:51:08 GMT
你好,可以查看下 log/ 目录下的相关日志有没有这样一段

2019-07-24 09:34:36,507 WARN  org.apache.flink.client.cli.CliFrontend
                - Could not load CLI class
org.apache.flink.yarn.cli.FlinkYarnSessionCli.

java.lang.NoClassDefFoundError:
org/apache/hadoop/yarn/exceptions/YarnException

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:264)

at
org.apache.flink.client.cli.CliFrontend.loadCustomCommandLine(CliFrontend.java:1187)

at
org.apache.flink.client.cli.CliFrontend.loadCustomCommandLines(CliFrontend.java:1147)

at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1072)

Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.exceptions.YarnException

at java.net.URLClassLoader.findClass(URLClassLoader.java:382)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 5 more


如果有的话,现在 Flink 把核心包和 hadoop 的 pre-bundled 包分开 release,需要你单独下载
pre-bundled 的
hadoop 然后放到 lib/ 文件夹里。


具体地,请仔细阅读下载页面[1] "Apache Flink 1.8.1" 字样上面的文字内容。

Best,
tison.

[1] https://flink.apache.org/downloads.html


王佩 <wangpei@cmcm.com> 于2019年7月24日周三 上午9:30写道:

> 之前下载的Flink 1.8,运行bin/flink run --help,会有 yarn-cluster 的一些参数,如下:
> Options for yarn-cluster mode:
>      -d,--detached                        If present, runs the job in
> detached
>                                           mode
>      -m,--jobmanager <arg>                Address of the JobManager
> (master) to
>                                           which to connect. Use this flag
> to
>                                           connect to a different JobManager
> than
>                                           the one specified in the
>                                           configuration.
>      -sae,--shutdownOnAttachedExit        If the job is submitted in
> attached
>                                           mode, perform a best-effort
> cluster
>                                           shutdown when the CLI is
> terminated
>                                           abruptly, e.g., in response to a
> user
>                                           interrupt, such as typing Ctrl +
> C.
>      -yD <property=value>                 use value for given property
>      -yd,--yarndetached                   If present, runs the job in
> detached
>                                           mode (deprecated; use non-YARN
>                                           specific option instead)
>      -yh,--yarnhelp                       Help for the Yarn session CLI.
>      -yid,--yarnapplicationId <arg>       Attach to running YARN session
>      -yj,--yarnjar <arg>                  Path to Flink jar file
>      -yjm,--yarnjobManagerMemory <arg>    Memory for JobManager Container
> with
>                                           optional unit (default: MB)
>      -yn,--yarncontainer <arg>            Number of YARN container to
> allocate
>                                           (=Number of Task Managers)
>      -ynl,--yarnnodeLabel <arg>           Specify YARN node label for the
> YARN
>                                           application
>      -ynm,--yarnname <arg>                Set a custom name for the
> application
>                                           on YARN
>      -yq,--yarnquery                      Display available YARN resources
>                                           (memory, cores)
>      -yqu,--yarnqueue <arg>               Specify YARN queue.
>      -ys,--yarnslots <arg>                Number of slots per TaskManager
>      -yst,--yarnstreaming                 Start Flink in streaming mode
>      -yt,--yarnship <arg>                 Ship files in the specified
> directory
>                                           (t for transfer)
>      -ytm,--yarntaskManagerMemory <arg>   Memory per TaskManager Container
> with
>                                           optional unit (default: MB)
>      -yz,--yarnzookeeperNamespace <arg>   Namespace to create the Zookeeper
>                                           sub-paths for high availability
> mode
>      -z,--zookeeperNamespace <arg>        Namespace to create the Zookeeper
>                                           sub-paths for high availability
> mode
>
>
> 现在下载的Flink 1.8,运行bin/flink run --help,总共只有如下参数,少了yarn-cluster选项:
> Action "run" compiles and runs a program.
>
>   Syntax: run [OPTIONS] <jar-file> <arguments>
>   "run" action options:
>      -c,--class <classname>               Class with the program entry
> point
>                                           ("main" method or "getPlan()"
> method.
>                                           Only needed if the JAR file does
> not
>                                           specify the class in its
> manifest.
>      -C,--classpath <url>                 Adds a URL to each user code
>                                           classloader  on all nodes in the
>                                           cluster. The paths must specify a
>                                           protocol (e.g. file://) and be
>                                           accessible on all nodes (e.g. by
> means
>                                           of a NFS share). You can use this
>                                           option multiple times for
> specifying
>                                           more than one URL. The protocol
> must
>                                           be supported by the {@link
>                                           java.net.URLClassLoader}.
>      -d,--detached                        If present, runs the job in
> detached
>                                           mode
>      -n,--allowNonRestoredState           Allow to skip savepoint state
> that
>                                           cannot be restored. You need to
> allow
>                                           this if you removed an operator
> from
>                                           your program that was part of the
>                                           program when the savepoint was
>                                           triggered.
>      -p,--parallelism <parallelism>       The parallelism with which to run
> the
>                                           program. Optional flag to
> override the
>                                           default value specified in the
>                                           configuration.
>      -q,--sysoutLogging                   If present, suppress logging
> output to
>                                           standard out.
>      -s,--fromSavepoint <savepointPath>   Path to a savepoint to restore
> the job
>                                           from (for example
>                                           hdfs:///flink/savepoint-1537).
>      -sae,--shutdownOnAttachedExit        If the job is submitted in
> attached
>                                           mode, perform a best-effort
> cluster
>                                           shutdown when the CLI is
> terminated
>                                           abruptly, e.g., in response to a
> user
>                                           interrupt, such as typing Ctrl +
> C.
>   Options for default mode:
>      -m,--jobmanager <arg>           Address of the JobManager (master) to
> which
>                                      to connect. Use this flag to connect
> to a
>                                      different JobManager than the one
> specified
>                                      in the configuration.
>      -z,--zookeeperNamespace <arg>   Namespace to create the Zookeeper
> sub-paths
>                                      for high availability mode
>
>
> 发现,少了yarn-cluster的一些参数,请教下,这是什么原因啊
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message