flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zili Chen <wander4...@gmail.com>
Subject Re: 回复: flink启动等待10分钟问题
Date Thu, 22 Aug 2019 05:03:42 GMT
基本上你是卡在上传用户 jar 这一步了,提交任务到部署成功是一瞬间的

2019-08-22 11:38:02,185 INFO
org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Submitting
application master application_1566383236573_0004
2019-08-22 11:38:02,226 INFO
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl         - Submitted
application application_1566383236573_0004

中间隔开的时间 FLINK 干的事情就是上传用户依赖的 jar 包

Best,
tison.


々守护々 <346531110@qq.com> 于2019年8月22日周四 上午11:49写道:

> 这个是终端打印的日志,十分钟后才启动- Submitting application master
> application_1566383236573_0004
>
>
>
> 2019-08-22 11:28:21,766 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - This YARN
> session requires 34816MB of memory in the cluster. There are currently only
> 26624MB available.
> The Flink YARN client will try to allocate the YARN session, but maybe not
> all TaskManagers are connecting from the beginning because the resources
> are currently not available in the cluster. The allocation might take more
> time than usual because the Flink YARN client needs to wait until the
> resources become available.
> 2019-08-22 11:28:21,766 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - There is
> not enough memory available in the YARN cluster. The TaskManager(s) require
> 8192MB each. NodeManagers available: [14336, 12288]
> After allocating the JobManager (2048MB) and (2/4) TaskManagers, the
> following NodeManagers are available: [4096, 4096]
> The Flink YARN client will try to allocate the YARN session, but maybe not
> all TaskManagers are connecting from the beginning because the resources
> are currently not available in the cluster. The allocation might take more
> time than usual because the Flink YARN client needs to wait until the
> resources become available.
> 2019-08-22 11:28:21,766 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - There is
> not enough memory available in the YARN cluster. The TaskManager(s) require
> 8192MB each. NodeManagers available: [14336, 12288]
> After allocating the JobManager (2048MB) and (3/4) TaskManagers, the
> following NodeManagers are available: [4096, 4096]
> The Flink YARN client will try to allocate the YARN session, but maybe not
> all TaskManagers are connecting from the beginning because the resources
> are currently not available in the cluster. The allocation might take more
> time than usual because the Flink YARN client needs to wait until the
> resources become available.
> 2019-08-22 11:28:21,767 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Cluster
> specification: ClusterSpecification{masterMemoryMB=2048,
> taskManagerMemoryMB=8192, numberTaskManagers=4, slotsPerTaskManager=2}
> 2019-08-22 11:28:22,317 WARN
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - The
> configuration directory ('/usr/flink/flink-1.8.1/conf') contains both LOG4J
> and Logback configuration files. Please delete or rename one of them.
> 2019-08-22 11:38:02,185 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Submitting
> application master application_1566383236573_0004
> 2019-08-22 11:38:02,226 INFO
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl         - Submitted
> application application_1566383236573_0004
> 2019-08-22 11:38:02,226 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Waiting for
> the cluster to be allocated
> 2019-08-22 11:38:02,228 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deploying
> cluster, current state ACCEPTED
> 2019-08-22 11:38:07,244 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - YARN
> application has been deployed successfully.
> 2019-08-22 11:38:07,244 INFO
> org.apache.flink.yarn.AbstractYarnClusterDescriptor           - The Flink
> YARN client has been started in detached mode. In order to stop Flink on
> YARN, use the following command or a YARN web interface to stop it:
> yarn application -kill application_1566383236573_0004
> Please also note that the temporary files of the YARN session in the home
> directory will not be removed.
> Job has been submitted with JobID c16c4fc1556ccb2ceaaa2f0e8f32ec88
>
> ------------------ 原始邮件 ------------------
> *发件人:* "Zili Chen"<wander4096@gmail.com>;
> *发送时间:* 2019年8月22日(星期四) 中午11:37
> *收件人:* "user-zh"<user-zh@flink.apache.org>;
> *主题:* Re: 回复: flink启动等待10分钟问题
>
> 你说的【停在那儿了】是说 flink run 的终端输出不动了吗?你看一下这个终端输出里 YARN 是什么时候 accept 你的应用的,我怀疑是
> YARN 集群忙导致 10 分钟没响应。
>
> Best,
> tison.
>
>
> Zili Chen <wander4096@gmail.com> 于2019年8月22日周四 上午11:35写道:
>
> > user-zh 不支持贴图,你用下第三方存储然后贴个链接吧,或者我记得可以传邮件附件
> >
> > Best,
> > tison.
> >
> >
> > 々守护々 <346531110@qq.com> 于2019年8月22日周四 上午11:33写道:
> >
> >> 是这样的,在启动的是后客户端日志就停在那了,等待10分钟后才能生成yarn的application
> >> <http://hadoop.changba.com/cluster/app/application_1566383236573_0003>
> >>
> >>
> >> ------------------ 原始邮件 ------------------
> >> *发件人:* "Yun Tang"<myasuka@live.com>;
> >> *发送时间:* 2019年8月22日(星期四) 中午11:23
> >> *收件人:* "user-zh"<user-zh@flink.apache.org>;
> >> *主题:* Re: 回复: flink启动等待10分钟问题
> >>
> >> 启动时间是 20:00:25,task都处于running甚至第一次checkpoint
> >> completed的时间是20:00:42,一共才17秒,何来10分钟的问题?
> >> ________________________________
> >> From: 々守护々 <346531110@qq.com>
> >> Sent: Thursday, August 22, 2019 11:18
> >> To: user-zh <user-zh@flink.apache.org>
> >> Subject: 回复: flink启动等待10分钟问题
> >>
> >> 您好,这个是我jobmanager启动日志,请帮忙看看,谢谢!
> >>
> >>
> >> 2019-08-21 20:00:25,428 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
> >>
> --------------------------------------------------------------------------------
> >> 2019-08-21 20:00:25,430 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
> Registered
> >> UNIX signal handlers for [TERM, HUP, INT]
> >> 2019-08-21 20:00:25,433 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - YARN
> daemon
> >> is running as: root Yarn client user obtainer: root
> >> 2019-08-21 20:00:25,437 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: high-availability.cluster-id,
> >> application_1566383236573_0003
> >> 2019-08-21 20:00:25,437 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: jobmanager.rpc.address, localhost
> >> 2019-08-21 20:00:25,437 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: jobmanager.rpc.port, 6123
> >> 2019-08-21 20:00:25,437 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: high-availability.zookeeper.path.root,
> /flink-yarn
> >> 2019-08-21 20:00:25,437 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: high-availability.storageDir,
> >> hdfs://host51:9000/flink/ha-yarn
> >> 2019-08-21 20:00:25,438 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: high-availability.zookeeper.quorum, host51:2181
> >> 2019-08-21 20:00:25,438 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: internal.cluster.execution-mode, DETACHED
> >> 2019-08-21 20:00:25,438 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: high-availability, zookeeper
> >> 2019-08-21 20:00:25,438 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: parallelism.default, 1
> >> 2019-08-21 20:00:25,438 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: yarn.application-attempts, 10
> >> 2019-08-21 20:00:25,438 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: taskmanager.numberOfTaskSlots, 2
> >> 2019-08-21 20:00:25,439 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: taskmanager.heap.size, 8192m
> >> 2019-08-21 20:00:25,439 INFO
> >> org.apache.flink.configuration.GlobalConfiguration            - Loading
> >> configuration property: jobmanager.heap.size, 2048m
> >> 2019-08-21 20:00:25,465 WARN
> >> org.apache.flink.configuration.Configuration                  - Config
> uses
> >> deprecated configuration key 'web.port' instead of proper key
> >> 'rest.bind-port'
> >> 2019-08-21 20:00:25,469 INFO
> >> org.apache.flink.runtime.clusterframework.BootstrapTools      - Setting
> >> directories for temporary files to:
> >>
> /usr/hadoop/hadoop-2.7.7/tmp/nm-local-dir/usercache/root/appcache/application_1566383236573_0003
> >> 2019-08-21 20:00:25,485 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting
> >> YarnJobClusterEntrypoint.
> >> 2019-08-21 20:00:25,485 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install
> >> default filesystem.
> >> 2019-08-21 20:00:25,561 INFO
> >> org.apache.flink.runtime.security.modules.HadoopModule        - Hadoop
> user
> >> set to root (auth:SIMPLE)
> >> 2019-08-21 20:00:25,580 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -
> >> Initializing cluster services.
> >> 2019-08-21 20:00:25,757 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils         - Trying
> to
> >> start actor system at host53:0
> >> 2019-08-21 20:00:26,187 INFO
> >> akka.event.slf4j.Slf4jLogger                                  -
> Slf4jLogger
> >> started
> >> 2019-08-21 20:00:26,251 INFO
> >> akka.remote.Remoting                                          - Starting
> >> remoting
> >> 2019-08-21 20:00:26,439 INFO
> >> akka.remote.Remoting                                          - Remoting
> >> started; listening on addresses :[akka.tcp://flink@host53:59609]
> >> 2019-08-21 20:00:26,447 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils         - Actor
> >> system started at akka.tcp://flink@host53:59609
> >> 2019-08-21 20:00:27,011 INFO
> >> org.apache.flink.runtime.blob.FileSystemBlobStore             - Creating
> >> highly available BLOB storage directory at
> >> hdfs://host51:9000/flink/ha-yarn/application_1566383236573_0003/blob
> >> 2019-08-21 20:00:27,097 INFO
> >> org.apache.flink.runtime.util.ZooKeeperUtils                  -
> Enforcing
> >> default ACL for ZK connections
> >> 2019-08-21 20:00:27,097 INFO
> >> org.apache.flink.runtime.util.ZooKeeperUtils                  - Using
> >> '/flink-yarn/application_1566383236573_0003' as Zookeeper namespace.
> >> 2019-08-21 20:00:27,161 INFO
> >>
> org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
> >> - Starting
> >> 2019-08-21 20:00:27,169 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >>
> environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f,
> >> built on 03/23/2017 10:13 GMT
> >> 2019-08-21 20:00:27,169 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:host.name=host53
> >> 2019-08-21 20:00:27,169 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:java.version=1.8.0_77
> >> 2019-08-21 20:00:27,169 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:java.vendor=Oracle Corporation
> >> 2019-08-21 20:00:27,169 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >>
> environment:java.library.path=:/usr/hadoop/hadoop-2.7.7/lib/native:/usr/hadoop/hadoop-2.7.7/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:java.io.tmpdir=/tmp
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:java.compiler=<NA>
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:os.name=Linux
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:os.arch=amd64
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:os.version=3.16.0-0.bpo.4-amd64
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:user.name=root
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >> environment:user.home=/root
> >> 2019-08-21 20:00:27,170 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> Client
> >>
> environment:user.dir=/usr/hadoop/hadoop-2.7.7/tmp/nm-local-dir/usercache/root/appcache/application_1566383236573_0003/container_1566383236573_0003_01_000001
> >> 2019-08-21 20:00:27,171 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper  -
> >> Initiating client connection, connectString=host51:2181
> >> sessionTimeout=60000
> >>
> watcher=org.apache.flink.shaded.curator.org.apache.curator.ConnectionState@319854f0
> >> 2019-08-21 20:00:27,184 WARN
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn  -
> SASL
> >> configuration failed: javax.security.auth.login.LoginException: No JAAS
> >> configuration section named 'Client' was found in specified JAAS
> >> configuration file: '/tmp/jaas-4274640356638794035.conf'. Will continue
> >> connection to Zookeeper server without SASL authentication, if Zookeeper
> >> server allows it.
> >> 2019-08-21 20:00:27,186 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn  -
> >> Opening socket connection to server host51/10.10.10.51:2181
> >> 2019-08-21 20:00:27,187 ERROR
> >> org.apache.flink.shaded.curator.org.apache.curator.ConnectionState  -
> >> Authentication failed
> >> 2019-08-21 20:00:27,187 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn  -
> Socket
> >> connection established to host51/10.10.10.51:2181, initiating session
> >> 2019-08-21 20:00:27,193 INFO
> >> org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn  -
> >> Session establishment complete on server host51/10.10.10.51:2181,
> >> sessionid = 0x1001a86de810008, negotiated timeout = 40000
> >> 2019-08-21 20:00:27,193 INFO
> >> org.apache.flink.runtime.blob.BlobServer                      - Created
> >> BLOB server storage directory
> >>
> /usr/hadoop/hadoop-2.7.7/tmp/nm-local-dir/usercache/root/appcache/application_1566383236573_0003/blobStore-eaa2bb2b-cddb-42e3-8ce5-bf2f83e8567a
> >> 2019-08-21 20:00:27,195 INFO
> >>
> org.apache.flink.shaded.curator.org.apache.curator.framework.state.ConnectionStateManager
> >> - State change: CONNECTED
> >> 2019-08-21 20:00:27,198 INFO
> >> org.apache.flink.runtime.blob.BlobServer                      - Started
> >> BLOB server at 0.0.0.0:58591 - max concurrent requests: 50 - max
> >> backlog: 1000
> >> 2019-08-21 20:00:27,213 INFO
> >> org.apache.flink.runtime.metrics.MetricRegistryImpl           - No
> metrics
> >> reporter configured, no metrics will be exposed/reported.
> >> 2019-08-21 20:00:27,215 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying
> to
> >> start actor system at host53:0
> >> 2019-08-21 20:00:27,239 INFO
> >> akka.event.slf4j.Slf4jLogger                                  -
> Slf4jLogger
> >> started
> >> 2019-08-21 20:00:27,247 INFO
> >> akka.remote.Remoting                                          - Starting
> >> remoting
> >> 2019-08-21 20:00:27,255 INFO
> >> akka.remote.Remoting                                          - Remoting
> >> started; listening on addresses :[akka.tcp://flink-metrics@host53
> :59941]
> >> 2019-08-21 20:00:27,257 INFO
> >> org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Actor
> >> system started at akka.tcp://flink-metrics@host53:59941
> >> 2019-08-21 20:00:27,263 INFO
> >> org.apache.flink.runtime.blob.TransientBlobCache              - Created
> >> BLOB cache storage directory
> >>
> /usr/hadoop/hadoop-2.7.7/tmp/nm-local-dir/usercache/root/appcache/application_1566383236573_0003/blobStore-405a4797-824f-48da-bb50-2ee2e230d249
> >> 2019-08-21 20:00:27,296 WARN
> >> org.apache.flink.configuration.Configuration                  - Config
> uses
> >> deprecated configuration key 'web.port' instead of proper key
> >> 'rest.bind-port'
> >> 2019-08-21 20:00:27,298 WARN
> >> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint  - Upload
> >> directory
> >> /tmp/flink-web-57c9cddd-64af-4e3d-96a9-4f9e78d20b86/flink-web-upload
> does
> >> not exist, or has been deleted externally. Previously uploaded files
> are no
> >> longer available.
> >> 2019-08-21 20:00:27,298 INFO
> >> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint  - Created
> >> directory
> >> /tmp/flink-web-57c9cddd-64af-4e3d-96a9-4f9e78d20b86/flink-web-upload for
> >> file uploads.
> >> 2019-08-21 20:00:27,320 INFO
> >> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint  -
> Starting
> >> rest endpoint.
> >> 2019-08-21 20:00:27,616 INFO
> >> org.apache.flink.runtime.webmonitor.WebMonitorUtils           -
> Determined
> >> location of main cluster component log file:
> >>
> /usr/hadoop/hadoop-2.7.7/logs/userlogs/application_1566383236573_0003/container_1566383236573_0003_01_000001/jobmanager.log
> >> 2019-08-21 20:00:27,616 INFO
> >> org.apache.flink.runtime.webmonitor.WebMonitorUtils           -
> Determined
> >> location of main cluster component stdout file:
> >>
> /usr/hadoop/hadoop-2.7.7/logs/userlogs/application_1566383236573_0003/container_1566383236573_0003_01_000001/jobmanager.out
> >> 2019-08-21 20:00:27,771 INFO
> >> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint  - Rest
> >> endpoint listening at host53:46401
> >> 2019-08-21 20:00:27,772 INFO
> >> org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
> -
> >> Starting ZooKeeperLeaderElectionService
> >> ZooKeeperLeaderElectionService{leaderPath='/leader/rest_server_lock'}.
> >> 2019-08-21 20:00:27,795 WARN
> >> org.apache.flink.shaded.curator.org.apache.curator.utils.ZKPaths  - The
> >> version of ZooKeeper being used doesn't support Container nodes.
> >> CreateMode.PERSISTENT will be used instead.
> >> 2019-08-21 20:00:27,807 INFO
> >> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint  - Web
> >> frontend listening at http://host53:46401.
> >> 2019-08-21 20:00:27,836 INFO
> >> org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint  -
> >> http://host53:46401 was granted leadership with
> >> leaderSessionID=8487757f-fd82-42b2-a1a6-adcc69e1eba1
> >> 2019-08-21 20:00:27,932 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting
> >> RPC endpoint for org.apache.flink.yarn.YarnResourceManager at
> >> akka://flink/user/resourcemanager .
> >> 2019-08-21 20:00:28,021 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting
> >> RPC endpoint for org.apache.flink.runtime.dispatcher.MiniDispatcher at
> >> akka://flink/user/dispatcher .
> >> 2019-08-21 20:00:28,033 INFO
> >>
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> >> Starting ZooKeeperLeaderRetrievalService /leader/resource_manager_lock.
> >> 2019-08-21 20:00:28,033 INFO
> >>
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> >> Starting ZooKeeperLeaderRetrievalService /leader/dispatcher_lock.
> >> 2019-08-21 20:00:28,033 INFO
> >> org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
> -
> >> Starting ZooKeeperLeaderElectionService
> >> ZooKeeperLeaderElectionService{leaderPath='/leader/dispatcher_lock'}.
> >> 2019-08-21 20:00:28,045 INFO
> >> org.apache.flink.runtime.dispatcher.MiniDispatcher            -
> Dispatcher
> >> akka.tcp://flink@host53:59609/user/dispatcher was granted leadership
> >> with fencing token 6216d17e-56b1-43bb-9bb3-f9fa5b8921be
> >> 2019-08-21 20:00:28,048 INFO
> >> org.apache.flink.runtime.dispatcher.MiniDispatcher            -
> Recovering
> >> all persisted jobs.
> >> 2019-08-21 20:00:28,054 INFO
> >> org.apache.hadoop.yarn.client.RMProxy                         -
> Connecting
> >> to ResourceManager at host51/10.10.10.51:8030
> >> 2019-08-21 20:00:28,078 INFO
> >> org.apache.flink.runtime.rpc.akka.AkkaRpcService              - Starting
> >> RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at
> >> akka://flink/user/jobmanager_0 .
> >> 2019-08-21 20:00:28,086 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  -
> >> Initializing job StreamKafkaMain (e5ea83238d721e932b4645589660406c).
> >> 2019-08-21 20:00:28,099 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  - Using
> >> restart strategy
> >> FixedDelayRestartStrategy(maxNumberRestartAttempts=2147483647,
> >> delayBetweenRestartAttempts=0) for StreamKafkaMain
> >> (e5ea83238d721e932b4645589660406c).
> >> 2019-08-21 20:00:28,122 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job
> >> recovers via failover strategy: full graph restart
> >> 2019-08-21 20:00:28,153 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  - Running
> >> initialization on master for job StreamKafkaMain
> >> (e5ea83238d721e932b4645589660406c).
> >> 2019-08-21 20:00:28,154 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  -
> >> Successfully ran initialization on master in 0 ms.
> >> 2019-08-21 20:00:28,187 INFO
> >> org.apache.flink.runtime.util.ZooKeeperUtils                  -
> Initialized
> >> ZooKeeperCompletedCheckpointStore in
> >> '/checkpoints/e5ea83238d721e932b4645589660406c'.
> >> 2019-08-21 20:00:28,196 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     -
> Recovered 0
> >> containers from previous attempts ([]).
> >> 2019-08-21 20:00:28,199 INFO
> >>
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  -
> >> yarn.client.max-cached-nodemanagers-proxies : 0
> >> 2019-08-21 20:00:28,200 INFO
> >> org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
> -
> >> Starting ZooKeeperLeaderElectionService
> >>
> ZooKeeperLeaderElectionService{leaderPath='/leader/resource_manager_lock'}.
> >> 2019-08-21 20:00:28,203 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  - No state
> >> backend has been configured, using default (Memory / JobManager)
> >> MemoryStateBackend (data in heap memory / checkpoints to JobManager)
> >> (checkpoints: 'null', savepoints: 'null', asynchronous: TRUE,
> maxStateSize:
> >> 5242880)
> >> 2019-08-21 20:00:28,211 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     -
> >> ResourceManager akka.tcp://flink@host53:59609/user/resourcemanager was
> >> granted leadership with fencing token 97537c0d61bbf091e37cb6c45b784568
> >> 2019-08-21 20:00:28,214 INFO
> >> org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager  -
> >> Starting the SlotManager.
> >> 2019-08-21 20:00:28,225 INFO
> >> org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore  -
> >> Recovering checkpoints from ZooKeeper.
> >> 2019-08-21 20:00:28,229 INFO
> >> org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore  -
> >> Found 0 checkpoints in ZooKeeper.
> >> 2019-08-21 20:00:28,229 INFO
> >> org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore  -
> >> Trying to fetch 0 checkpoints from storage.
> >> 2019-08-21 20:00:28,233 INFO
> >> org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
> -
> >> Starting ZooKeeperLeaderElectionService
> >>
> ZooKeeperLeaderElectionService{leaderPath='/leader/e5ea83238d721e932b4645589660406c/job_manager_lock'}.
> >> 2019-08-21 20:00:28,248 INFO
> >> org.apache.flink.runtime.jobmaster.JobManagerRunner           -
> JobManager
> >> runner for job StreamKafkaMain (e5ea83238d721e932b4645589660406c) was
> >> granted leadership with session id 12c96c94-ebbd-4991-a457-6f0320d71bc1
> at
> >> akka.tcp://flink@host53:59609/user/jobmanager_0.
> >> 2019-08-21 20:00:28,260 INFO
> >>
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> >> Starting ZooKeeperLeaderRetrievalService /leader/resource_manager_lock.
> >> 2019-08-21 20:00:28,260 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  - Starting
> >> execution of job StreamKafkaMain (e5ea83238d721e932b4645589660406c)
> under
> >> job master id a4576f0320d71bc112c96c94ebbd4991.
> >> 2019-08-21 20:00:28,261 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Job
> >> StreamKafkaMain (e5ea83238d721e932b4645589660406c) switched from state
> >> CREATED to RUNNING.
> >> 2019-08-21 20:00:28,266 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (1/4)
> >> (ee39866835f7b7387f518e352787a532) switched from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,280 INFO
> >> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      - Cannot
> >> serve slot request, no ResourceManager connected. Adding as pending
> request
> >> [SlotRequestId{f6123820ffb8b3e906b876e9dea4497b}]
> >> 2019-08-21 20:00:28,287 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (2/4)
> >> (f1a893fc5e2440331c8e6db1874ad220) switched from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,288 INFO
> >> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      - Cannot
> >> serve slot request, no ResourceManager connected. Adding as pending
> request
> >> [SlotRequestId{fdbb2e71c9bc795452687396abff7bf8}]
> >> 2019-08-21 20:00:28,288 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (3/4)
> >> (4756a0450881325af5fb396a18dbb8ae) switched from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,289 INFO
> >> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      - Cannot
> >> serve slot request, no ResourceManager connected. Adding as pending
> request
> >> [SlotRequestId{13970fab9312acf9d5063b1d1a33aa0e}]
> >> 2019-08-21 20:00:28,289 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (4/4)
> >> (78f520582607e26b365fca483fc98d4c) switched from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,289 INFO
> >> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      - Cannot
> >> serve slot request, no ResourceManager connected. Adding as pending
> request
> >> [SlotRequestId{0621470f796aa3ea4291d1b8b423e428}]
> >> 2019-08-21 20:00:28,289 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - window:
> >> (TumblingGroupWindow('w$, 'rowtime, 60000.millis)), select: (COUNT(*) AS
> >> pv, COUNT(DISTINCT curuserid) AS uv, start('w$) AS w$start, end('w$) AS
> >> w$end, rowtime('w$) AS w$rowtime, proctime('w$) AS w$proctime) ->
> select:
> >> (pv, uv, utc2local(w$end) AS EXPR$2) -> to: Row (1/1)
> >> (0071e1c7c4f70646914329f81dcbd349) switched from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,291 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (1/4) (0a206444b10553cce42fa34ea0b15497) switched from
> CREATED
> >> to SCHEDULED.
> >> 2019-08-21 20:00:28,291 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (2/4) (a5d1bb759ee6c8001313fe39c7982a8b) switched from
> CREATED
> >> to SCHEDULED.
> >> 2019-08-21 20:00:28,291 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (3/4) (9543bbde6538ea9d407af3012fede5c8) switched from
> CREATED
> >> to SCHEDULED.
> >> 2019-08-21 20:00:28,291 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (4/4) (0e8c4210bd81e581fbf354b86397ce41) switched from
> CREATED
> >> to SCHEDULED.
> >> 2019-08-21 20:00:28,291 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (1/4) (39d437bceb033f5bd37747e9142c7f0e)
> switched
> >> from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,291 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (2/4) (aa4f64edc4f10da2e8bdbf6ebf0aeb83)
> switched
> >> from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,291 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (3/4) (f109918ce195d1c88c2c63a44934217f)
> switched
> >> from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,292 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (4/4) (9aa5bb28f3ba75afc1de47bd9a4e5d01)
> switched
> >> from CREATED to SCHEDULED.
> >> 2019-08-21 20:00:28,295 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  -
> Connecting
> >> to ResourceManager akka.tcp://flink@host53
> >> :59609/user/resourcemanager(97537c0d61bbf091e37cb6c45b784568)
> >> 2019-08-21 20:00:28,300 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  - Resolved
> >> ResourceManager address, beginning registration
> >> 2019-08-21 20:00:28,300 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  -
> >> Registration at ResourceManager attempt 1 (timeout=100ms)
> >> 2019-08-21 20:00:28,304 INFO
> >>
> org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService  -
> >> Starting ZooKeeperLeaderRetrievalService
> >> /leader/e5ea83238d721e932b4645589660406c/job_manager_lock.
> >> 2019-08-21 20:00:28,304 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     -
> Registering
> >> job manager a4576f0320d71bc112c96c94ebbd4991@akka.tcp://flink@host53
> :59609/user/jobmanager_0
> >> for job e5ea83238d721e932b4645589660406c.
> >> 2019-08-21 20:00:28,310 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     -
> Registered
> >> job manager a4576f0320d71bc112c96c94ebbd4991@akka.tcp://flink@host53
> :59609/user/jobmanager_0
> >> for job e5ea83238d721e932b4645589660406c.
> >> 2019-08-21 20:00:28,313 INFO
> >> org.apache.flink.runtime.jobmaster.JobMaster                  -
> JobManager
> >> successfully registered at ResourceManager, leader id:
> >> 97537c0d61bbf091e37cb6c45b784568.
> >> 2019-08-21 20:00:28,314 INFO
> >> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      -
> Requesting
> >> new slot [SlotRequestId{13970fab9312acf9d5063b1d1a33aa0e}] and profile
> >> ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0,
> >> nativeMemoryInMB=0, networkMemoryInMB=0} from resource manager.
> >> 2019-08-21 20:00:28,315 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Request
> >> slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1,
> >> directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} for job
> >> e5ea83238d721e932b4645589660406c with allocation id
> >> faa6a521758e5be4b3dc1955ca0f51b4.
> >> 2019-08-21 20:00:28,316 INFO
> >> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      -
> Requesting
> >> new slot [SlotRequestId{fdbb2e71c9bc795452687396abff7bf8}] and profile
> >> ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0,
> >> nativeMemoryInMB=0, networkMemoryInMB=0} from resource manager.
> >> 2019-08-21 20:00:28,316 INFO
> >> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      -
> Requesting
> >> new slot [SlotRequestId{0621470f796aa3ea4291d1b8b423e428}] and profile
> >> ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0,
> >> nativeMemoryInMB=0, networkMemoryInMB=0} from resource manager.
> >> 2019-08-21 20:00:28,316 INFO
> >> org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl      -
> Requesting
> >> new slot [SlotRequestId{f6123820ffb8b3e906b876e9dea4497b}] and profile
> >> ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0,
> >> nativeMemoryInMB=0, networkMemoryInMB=0} from resource manager.
> >> 2019-08-21 20:00:28,321 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     -
> Requesting
> >> new TaskExecutor container with resources <memory:8192, vCores:2>.
> Number
> >> pending requests 1.
> >> 2019-08-21 20:00:28,322 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Request
> >> slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1,
> >> directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} for job
> >> e5ea83238d721e932b4645589660406c with allocation id
> >> a2b419aea4a373686eeed6c46163a528.
> >> 2019-08-21 20:00:28,323 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Request
> >> slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1,
> >> directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} for job
> >> e5ea83238d721e932b4645589660406c with allocation id
> >> cbc11737ce6729cfe45f24c43180b5c8.
> >> 2019-08-21 20:00:28,323 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     -
> Requesting
> >> new TaskExecutor container with resources <memory:8192, vCores:2>.
> Number
> >> pending requests 2.
> >> 2019-08-21 20:00:28,323 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Request
> >> slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1,
> >> directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} for job
> >> e5ea83238d721e932b4645589660406c with allocation id
> >> f6566567c173a55b2bc52b72bcb24ff8.
> >> 2019-08-21 20:00:31,597 INFO
> >> org.apache.flink.runtime.checkpoint.CheckpointCoordinator     -
> Checkpoint
> >> triggering task Source: Custom Source -> Flat Map -> Filter -> Filter ->
> >> Map -> Timestamps/Watermarks -> from: (request, curuserid, timelong,
> >> rowtime) -> select: (rowtime, 0 AS $f1, curuserid) -> time attribute:
> >> (rowtime) (1/4) of job e5ea83238d721e932b4645589660406c is not in state
> >> RUNNING but SCHEDULED instead. Aborting checkpoint.
> >> 2019-08-21 20:00:33,749 INFO
> >> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Received
> >> new token for : host53:59938
> >> 2019-08-21 20:00:33,749 INFO
> >> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl         - Received
> >> new token for : host52:33246
> >> 2019-08-21 20:00:33,765 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Received
> >> new container: container_1566383236573_0003_01_000002 - Remaining
> pending
> >> container requests: 2
> >> 2019-08-21 20:00:33,765 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Removing
> >> container request Capability[<memory:8192, vCores:2>]Priority[1].
> Pending
> >> container requests 1.
> >> 2019-08-21 20:00:34,009 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Creating
> >> container launch context for TaskManagers
> >> 2019-08-21 20:00:34,010 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Starting
> >> TaskManagers
> >> 2019-08-21 20:00:34,021 INFO
> >>
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  -
> >> Opening proxy : host53:59938
> >> 2019-08-21 20:00:34,066 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Received
> >> new container: container_1566383236573_0003_01_000003 - Remaining
> pending
> >> container requests: 1
> >> 2019-08-21 20:00:34,066 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Removing
> >> container request Capability[<memory:8192, vCores:2>]Priority[1].
> Pending
> >> container requests 0.
> >> 2019-08-21 20:00:34,088 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Creating
> >> container launch context for TaskManagers
> >> 2019-08-21 20:00:34,089 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     - Starting
> >> TaskManagers
> >> 2019-08-21 20:00:34,090 INFO
> >>
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy  -
> >> Opening proxy : host52:33246
> >> 2019-08-21 20:00:36,597 INFO
> >> org.apache.flink.runtime.checkpoint.CheckpointCoordinator     -
> Checkpoint
> >> triggering task Source: Custom Source -> Flat Map -> Filter -> Filter ->
> >> Map -> Timestamps/Watermarks -> from: (request, curuserid, timelong,
> >> rowtime) -> select: (rowtime, 0 AS $f1, curuserid) -> time attribute:
> >> (rowtime) (1/4) of job e5ea83238d721e932b4645589660406c is not in state
> >> RUNNING but SCHEDULED instead. Aborting checkpoint.
> >> 2019-08-21 20:00:37,856 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     -
> Registering
> >> TaskManager with ResourceID container_1566383236573_0003_01_000002
> >> (akka.tcp://flink@host53:34608/user/taskmanager_0) at ResourceManager
> >> 2019-08-21 20:00:39,573 INFO
> >> org.apache.flink.yarn.YarnResourceManager                     -
> Registering
> >> TaskManager with ResourceID container_1566383236573_0003_01_000003
> >> (akka.tcp://flink@host52:32931/user/taskmanager_0) at ResourceManager
> >> 2019-08-21 20:00:39,659 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (1/4)
> >> (ee39866835f7b7387f518e352787a532) switched from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,659 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Source: Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (1/4)
> >> (attempt #0) to container_1566383236573_0003_01_000002 @ host53
> >> (dataPort=52456)
> >> 2019-08-21 20:00:39,664 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (2/4)
> >> (f1a893fc5e2440331c8e6db1874ad220) switched from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,664 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Source: Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (2/4)
> >> (attempt #0) to container_1566383236573_0003_01_000002 @ host53
> >> (dataPort=52456)
> >> 2019-08-21 20:00:39,665 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (3/4)
> >> (4756a0450881325af5fb396a18dbb8ae) switched from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,666 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Source: Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (3/4)
> >> (attempt #0) to container_1566383236573_0003_01_000003 @ host52
> >> (dataPort=39650)
> >> 2019-08-21 20:00:39,666 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (4/4)
> >> (78f520582607e26b365fca483fc98d4c) switched from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,666 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Source: Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (4/4)
> >> (attempt #0) to container_1566383236573_0003_01_000003 @ host52
> >> (dataPort=39650)
> >> 2019-08-21 20:00:39,666 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - window:
> >> (TumblingGroupWindow('w$, 'rowtime, 60000.millis)), select: (COUNT(*) AS
> >> pv, COUNT(DISTINCT curuserid) AS uv, start('w$) AS w$start, end('w$) AS
> >> w$end, rowtime('w$) AS w$rowtime, proctime('w$) AS w$proctime) ->
> select:
> >> (pv, uv, utc2local(w$end) AS EXPR$2) -> to: Row (1/1)
> >> (0071e1c7c4f70646914329f81dcbd349) switched from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,666 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> window: (TumblingGroupWindow('w$, 'rowtime, 60000.millis)), select:
> >> (COUNT(*) AS pv, COUNT(DISTINCT curuserid) AS uv, start('w$) AS w$start,
> >> end('w$) AS w$end, rowtime('w$) AS w$rowtime, proctime('w$) AS
> w$proctime)
> >> -> select: (pv, uv, utc2local(w$end) AS EXPR$2) -> to: Row (1/1)
> (attempt
> >> #0) to container_1566383236573_0003_01_000002 @ host53 (dataPort=52456)
> >> 2019-08-21 20:00:39,670 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (1/4) (0a206444b10553cce42fa34ea0b15497) switched from
> >> SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,670 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Sink: Print to Std. Out (1/4) (attempt #0) to
> >> container_1566383236573_0003_01_000003 @ host52 (dataPort=39650)
> >> 2019-08-21 20:00:39,671 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (2/4) (a5d1bb759ee6c8001313fe39c7982a8b) switched from
> >> SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,672 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Sink: Print to Std. Out (2/4) (attempt #0) to
> >> container_1566383236573_0003_01_000003 @ host52 (dataPort=39650)
> >> 2019-08-21 20:00:39,672 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (3/4) (9543bbde6538ea9d407af3012fede5c8) switched from
> >> SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,672 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Sink: Print to Std. Out (3/4) (attempt #0) to
> >> container_1566383236573_0003_01_000002 @ host53 (dataPort=52456)
> >> 2019-08-21 20:00:39,672 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (4/4) (0e8c4210bd81e581fbf354b86397ce41) switched from
> >> SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,673 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Sink: Print to Std. Out (4/4) (attempt #0) to
> >> container_1566383236573_0003_01_000002 @ host53 (dataPort=52456)
> >> 2019-08-21 20:00:39,673 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (1/4) (39d437bceb033f5bd37747e9142c7f0e)
> switched
> >> from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,673 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Map -> Sink: Print to Std. Out (1/4) (attempt #0) to
> >> container_1566383236573_0003_01_000003 @ host52 (dataPort=39650)
> >> 2019-08-21 20:00:39,675 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (2/4) (aa4f64edc4f10da2e8bdbf6ebf0aeb83)
> switched
> >> from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,675 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Map -> Sink: Print to Std. Out (2/4) (attempt #0) to
> >> container_1566383236573_0003_01_000003 @ host52 (dataPort=39650)
> >> 2019-08-21 20:00:39,675 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (3/4) (f109918ce195d1c88c2c63a44934217f)
> switched
> >> from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,675 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Map -> Sink: Print to Std. Out (3/4) (attempt #0) to
> >> container_1566383236573_0003_01_000002 @ host53 (dataPort=52456)
> >> 2019-08-21 20:00:39,676 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (4/4) (9aa5bb28f3ba75afc1de47bd9a4e5d01)
> switched
> >> from SCHEDULED to DEPLOYING.
> >> 2019-08-21 20:00:39,676 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        -
> Deploying
> >> Map -> Sink: Print to Std. Out (4/4) (attempt #0) to
> >> container_1566383236573_0003_01_000002 @ host53 (dataPort=52456)
> >> 2019-08-21 20:00:39,782 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (4/4) (0e8c4210bd81e581fbf354b86397ce41) switched from
> >> DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,783 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (3/4) (9543bbde6538ea9d407af3012fede5c8) switched from
> >> DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,784 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (4/4) (9aa5bb28f3ba75afc1de47bd9a4e5d01)
> switched
> >> from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,787 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (3/4) (f109918ce195d1c88c2c63a44934217f)
> switched
> >> from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,806 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (2/4)
> >> (f1a893fc5e2440331c8e6db1874ad220) switched from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,809 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (1/4)
> >> (ee39866835f7b7387f518e352787a532) switched from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,809 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - window:
> >> (TumblingGroupWindow('w$, 'rowtime, 60000.millis)), select: (COUNT(*) AS
> >> pv, COUNT(DISTINCT curuserid) AS uv, start('w$) AS w$start, end('w$) AS
> >> w$end, rowtime('w$) AS w$rowtime, proctime('w$) AS w$proctime) ->
> select:
> >> (pv, uv, utc2local(w$end) AS EXPR$2) -> to: Row (1/1)
> >> (0071e1c7c4f70646914329f81dcbd349) switched from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,991 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (1/4) (0a206444b10553cce42fa34ea0b15497) switched from
> >> DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,992 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Sink:
> Print
> >> to Std. Out (2/4) (a5d1bb759ee6c8001313fe39c7982a8b) switched from
> >> DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,996 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (1/4) (39d437bceb033f5bd37747e9142c7f0e)
> switched
> >> from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:39,997 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Map ->
> >> Sink: Print to Std. Out (2/4) (aa4f64edc4f10da2e8bdbf6ebf0aeb83)
> switched
> >> from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:40,005 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (4/4)
> >> (78f520582607e26b365fca483fc98d4c) switched from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:40,006 INFO
> >> org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source:
> >> Custom Source -> Flat Map -> Filter -> Filter -> Map ->
> >> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
> ->
> >> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime)
> (3/4)
> >> (4756a0450881325af5fb396a18dbb8ae) switched from DEPLOYING to RUNNING.
> >> 2019-08-21 20:00:41,605 INFO
> >> org.apache.flink.runtime.checkpoint.CheckpointCoordinator     -
> Triggering
> >> checkpoint 1 @ 1566388841597 for job e5ea83238d721e932b4645589660406c.
> >> 2019-08-21 20:00:42,466 INFO
> >> org.apache.flink.runtime.checkpoint.CheckpointCoordinator     -
> Completed
> >> checkpoint 1 for job e5ea83238d721e932b4645589660406c (7994 bytes in 821
> >> ms).
> >>
> >>
> >>
> >> ------------------ 原始邮件 ------------------
> >> 发件人: "Yun Tang"<myasuka@live.com>;
> >> 发送时间: 2019年8月22日(星期四) 中午11:08
> >> 收件人: "user-zh"<user-zh@flink.apache.org>;
> >>
> >> 主题: Re: flink启动等待10分钟问题
> >>
> >>
> >>
> >> Hi
> >>
> >> Flink on YARN作业启动时间长,有很多原因,例如资源不够在等待,container申请的时候又退出了。默认的slot
> >> request的timeout时间是5min,感觉你的作业应该是可能遇到了一个slot request
> >> timeout,然后又重新申请。最好能提供一下jobmanager的日志才好进一步分析。
> >>
> >> 祝好
> >> 唐云
> >> ________________________________
> >> From: 々守护々 <346531110@qq.com>
> >> Sent: Thursday, August 22, 2019 11:04
> >> To: user-zh <user-zh@flink.apache.org>
> >> Subject: flink启动等待10分钟问题
> >>
> >> 各位Flink社区大佬,您好!
> >>       我使用Flink SQL (Flink 1.8.1),在hadoop 集群中调整了一下yarn-site.xml文件的内存参数:
> >>         <property>
> >>                 <name>yarn.nodemanager.resource.memory-mb</name>
> >>                 <value>16384</value>
> >>         </property>
> >>         <property>
> >>                 <name>yarn.scheduler.minimum-allocation-mb</name>
> >>                 <value>1024</value>
> >>         </property>
> >>         <property>
> >>                 <name>yarn.scheduler.maximum-allocation-mb</name>
> >>                 <value>8192</value>
> >>         </property>
> >>
> >>
> >>
> >> 然后我去启动flink on yarn任务,启动参数:./flink-1.8.1/bin/flink run -m yarn-cluster
> -p
> >> 4 -d -c streaming.StreamKafkaMain -yn 4 -ys 2 -yjm 2048 -ytm 8192  -ynm
> >> bigdata6 ./flink_bigdata6.jar
> >> 最终能能启动成功,就是启动要等待10分钟,这个等待时间太长了,请问各位大佬有什么解决的方法吗?
> >>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message