flink-user-zh mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "々守护々" <346531...@qq.com>
Subject 回复: flink启动等待10分钟问题
Date Thu, 22 Aug 2019 03:18:22 GMT
您好,这个是我jobmanager启动日志,请帮忙看看,谢谢!


2019-08-21 20:00:25,428 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint      
  - --------------------------------------------------------------------------------
2019-08-21 20:00:25,430 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint      
  - Registered UNIX signal handlers for [TERM, HUP, INT]
2019-08-21 20:00:25,433 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint      
  - YARN daemon is running as: root Yarn client user obtainer: root
2019-08-21 20:00:25,437 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: high-availability.cluster-id, application_1566383236573_0003
2019-08-21 20:00:25,437 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: jobmanager.rpc.address, localhost
2019-08-21 20:00:25,437 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: jobmanager.rpc.port, 6123
2019-08-21 20:00:25,437 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: high-availability.zookeeper.path.root, /flink-yarn
2019-08-21 20:00:25,437 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: high-availability.storageDir, hdfs://host51:9000/flink/ha-yarn
2019-08-21 20:00:25,438 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: high-availability.zookeeper.quorum, host51:2181
2019-08-21 20:00:25,438 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: internal.cluster.execution-mode, DETACHED
2019-08-21 20:00:25,438 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: high-availability, zookeeper
2019-08-21 20:00:25,438 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: parallelism.default, 1
2019-08-21 20:00:25,438 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: yarn.application-attempts, 10
2019-08-21 20:00:25,438 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: taskmanager.numberOfTaskSlots, 2
2019-08-21 20:00:25,439 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: taskmanager.heap.size, 8192m
2019-08-21 20:00:25,439 INFO  org.apache.flink.configuration.GlobalConfiguration         
  - Loading configuration property: jobmanager.heap.size, 2048m
2019-08-21 20:00:25,465 WARN  org.apache.flink.configuration.Configuration               
  - Config uses deprecated configuration key 'web.port' instead of proper key 'rest.bind-port'
2019-08-21 20:00:25,469 INFO  org.apache.flink.runtime.clusterframework.BootstrapTools   
  - Setting directories for temporary files to: /usr/hadoop/hadoop-2.7.7/tmp/nm-local-dir/usercache/root/appcache/application_1566383236573_0003
2019-08-21 20:00:25,485 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint      
  - Starting YarnJobClusterEntrypoint.
2019-08-21 20:00:25,485 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint      
  - Install default filesystem.
2019-08-21 20:00:25,561 INFO  org.apache.flink.runtime.security.modules.HadoopModule     
  - Hadoop user set to root (auth:SIMPLE)
2019-08-21 20:00:25,580 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint      
  - Initializing cluster services.
2019-08-21 20:00:25,757 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils      
  - Trying to start actor system at host53:0
2019-08-21 20:00:26,187 INFO  akka.event.slf4j.Slf4jLogger                               
  - Slf4jLogger started
2019-08-21 20:00:26,251 INFO  akka.remote.Remoting                                       
  - Starting remoting
2019-08-21 20:00:26,439 INFO  akka.remote.Remoting                                       
  - Remoting started; listening on addresses :[akka.tcp://flink@host53:59609]
2019-08-21 20:00:26,447 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils      
  - Actor system started at akka.tcp://flink@host53:59609
2019-08-21 20:00:27,011 INFO  org.apache.flink.runtime.blob.FileSystemBlobStore          
  - Creating highly available BLOB storage directory at hdfs://host51:9000/flink/ha-yarn/application_1566383236573_0003/blob
2019-08-21 20:00:27,097 INFO  org.apache.flink.runtime.util.ZooKeeperUtils               
  - Enforcing default ACL for ZK connections
2019-08-21 20:00:27,097 INFO  org.apache.flink.runtime.util.ZooKeeperUtils               
  - Using '/flink-yarn/application_1566383236573_0003' as Zookeeper namespace.
2019-08-21 20:00:27,161 INFO  org.apache.flink.shaded.curator.org.apache.curator.framework.imps.CuratorFrameworkImpl
 - Starting
2019-08-21 20:00:27,169 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built
on 03/23/2017 10:13 GMT
2019-08-21 20:00:27,169 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:host.name=host53
2019-08-21 20:00:27,169 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:java.version=1.8.0_77
2019-08-21 20:00:27,169 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:java.vendor=Oracle Corporation
2019-08-21 20:00:27,169 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client 
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:java.library.path=:/usr/hadoop/hadoop-2.7.7/lib/native:/usr/hadoop/hadoop-2.7.7/lib/native:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:java.io.tmpdir=/tmp
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:java.compiler=<NA>
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:os.name=Linux
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:os.arch=amd64
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:os.version=3.16.0-0.bpo.4-amd64
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:user.name=root
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:user.home=/root
2019-08-21 20:00:27,170 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Client environment:user.dir=/usr/hadoop/hadoop-2.7.7/tmp/nm-local-dir/usercache/root/appcache/application_1566383236573_0003/container_1566383236573_0003_01_000001
2019-08-21 20:00:27,171 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ZooKeeper
 - Initiating client connection, connectString=host51:2181 sessionTimeout=60000 watcher=org.apache.flink.shaded.curator.org.apache.curator.ConnectionState@319854f0
2019-08-21 20:00:27,184 WARN  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn
 - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration
section named 'Client' was found in specified JAAS configuration file: '/tmp/jaas-4274640356638794035.conf'.
Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server
allows it.
2019-08-21 20:00:27,186 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn
 - Opening socket connection to server host51/10.10.10.51:2181
2019-08-21 20:00:27,187 ERROR org.apache.flink.shaded.curator.org.apache.curator.ConnectionState
 - Authentication failed
2019-08-21 20:00:27,187 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn
 - Socket connection established to host51/10.10.10.51:2181, initiating session
2019-08-21 20:00:27,193 INFO  org.apache.flink.shaded.zookeeper.org.apache.zookeeper.ClientCnxn
 - Session establishment complete on server host51/10.10.10.51:2181, sessionid = 0x1001a86de810008,
negotiated timeout = 40000
2019-08-21 20:00:27,193 INFO  org.apache.flink.runtime.blob.BlobServer                   
  - Created BLOB server storage directory /usr/hadoop/hadoop-2.7.7/tmp/nm-local-dir/usercache/root/appcache/application_1566383236573_0003/blobStore-eaa2bb2b-cddb-42e3-8ce5-bf2f83e8567a
2019-08-21 20:00:27,195 INFO  org.apache.flink.shaded.curator.org.apache.curator.framework.state.ConnectionStateManager
 - State change: CONNECTED
2019-08-21 20:00:27,198 INFO  org.apache.flink.runtime.blob.BlobServer                   
  - Started BLOB server at 0.0.0.0:58591 - max concurrent requests: 50 - max backlog: 1000
2019-08-21 20:00:27,213 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl        
  - No metrics reporter configured, no metrics will be exposed/reported.
2019-08-21 20:00:27,215 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint      
  - Trying to start actor system at host53:0
2019-08-21 20:00:27,239 INFO  akka.event.slf4j.Slf4jLogger                               
  - Slf4jLogger started
2019-08-21 20:00:27,247 INFO  akka.remote.Remoting                                       
  - Starting remoting
2019-08-21 20:00:27,255 INFO  akka.remote.Remoting                                       
  - Remoting started; listening on addresses :[akka.tcp://flink-metrics@host53:59941]
2019-08-21 20:00:27,257 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint      
  - Actor system started at akka.tcp://flink-metrics@host53:59941
2019-08-21 20:00:27,263 INFO  org.apache.flink.runtime.blob.TransientBlobCache           
  - Created BLOB cache storage directory /usr/hadoop/hadoop-2.7.7/tmp/nm-local-dir/usercache/root/appcache/application_1566383236573_0003/blobStore-405a4797-824f-48da-bb50-2ee2e230d249
2019-08-21 20:00:27,296 WARN  org.apache.flink.configuration.Configuration               
  - Config uses deprecated configuration key 'web.port' instead of proper key 'rest.bind-port'
2019-08-21 20:00:27,298 WARN  org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint
 - Upload directory /tmp/flink-web-57c9cddd-64af-4e3d-96a9-4f9e78d20b86/flink-web-upload does
not exist, or has been deleted externally. Previously uploaded files are no longer available.
2019-08-21 20:00:27,298 INFO  org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint
 - Created directory /tmp/flink-web-57c9cddd-64af-4e3d-96a9-4f9e78d20b86/flink-web-upload
for file uploads.
2019-08-21 20:00:27,320 INFO  org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint
 - Starting rest endpoint.
2019-08-21 20:00:27,616 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils        
  - Determined location of main cluster component log file: /usr/hadoop/hadoop-2.7.7/logs/userlogs/application_1566383236573_0003/container_1566383236573_0003_01_000001/jobmanager.log
2019-08-21 20:00:27,616 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils        
  - Determined location of main cluster component stdout file: /usr/hadoop/hadoop-2.7.7/logs/userlogs/application_1566383236573_0003/container_1566383236573_0003_01_000001/jobmanager.out
2019-08-21 20:00:27,771 INFO  org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint
 - Rest endpoint listening at host53:46401
2019-08-21 20:00:27,772 INFO  org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
 - Starting ZooKeeperLeaderElectionService ZooKeeperLeaderElectionService{leaderPath='/leader/rest_server_lock'}.
2019-08-21 20:00:27,795 WARN  org.apache.flink.shaded.curator.org.apache.curator.utils.ZKPaths
 - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT
will be used instead.
2019-08-21 20:00:27,807 INFO  org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint
 - Web frontend listening at http://host53:46401.
2019-08-21 20:00:27,836 INFO  org.apache.flink.runtime.jobmaster.MiniDispatcherRestEndpoint
 - http://host53:46401 was granted leadership with leaderSessionID=8487757f-fd82-42b2-a1a6-adcc69e1eba1
2019-08-21 20:00:27,932 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService           
  - Starting RPC endpoint for org.apache.flink.yarn.YarnResourceManager at akka://flink/user/resourcemanager
.
2019-08-21 20:00:28,021 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService           
  - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.MiniDispatcher at akka://flink/user/dispatcher
.
2019-08-21 20:00:28,033 INFO  org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService
 - Starting ZooKeeperLeaderRetrievalService /leader/resource_manager_lock.
2019-08-21 20:00:28,033 INFO  org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService
 - Starting ZooKeeperLeaderRetrievalService /leader/dispatcher_lock.
2019-08-21 20:00:28,033 INFO  org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
 - Starting ZooKeeperLeaderElectionService ZooKeeperLeaderElectionService{leaderPath='/leader/dispatcher_lock'}.
2019-08-21 20:00:28,045 INFO  org.apache.flink.runtime.dispatcher.MiniDispatcher         
  - Dispatcher akka.tcp://flink@host53:59609/user/dispatcher was granted leadership with fencing
token 6216d17e-56b1-43bb-9bb3-f9fa5b8921be
2019-08-21 20:00:28,048 INFO  org.apache.flink.runtime.dispatcher.MiniDispatcher         
  - Recovering all persisted jobs.
2019-08-21 20:00:28,054 INFO  org.apache.hadoop.yarn.client.RMProxy                      
  - Connecting to ResourceManager at host51/10.10.10.51:8030
2019-08-21 20:00:28,078 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService           
  - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/jobmanager_0
.
2019-08-21 20:00:28,086 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - Initializing job StreamKafkaMain (e5ea83238d721e932b4645589660406c).
2019-08-21 20:00:28,099 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - Using restart strategy FixedDelayRestartStrategy(maxNumberRestartAttempts=2147483647,
delayBetweenRestartAttempts=0) for StreamKafkaMain (e5ea83238d721e932b4645589660406c).
2019-08-21 20:00:28,122 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Job recovers via failover strategy: full graph restart
2019-08-21 20:00:28,153 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - Running initialization on master for job StreamKafkaMain (e5ea83238d721e932b4645589660406c).
2019-08-21 20:00:28,154 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - Successfully ran initialization on master in 0 ms.
2019-08-21 20:00:28,187 INFO  org.apache.flink.runtime.util.ZooKeeperUtils               
  - Initialized ZooKeeperCompletedCheckpointStore in '/checkpoints/e5ea83238d721e932b4645589660406c'.
2019-08-21 20:00:28,196 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Recovered 0 containers from previous attempts ([]).
2019-08-21 20:00:28,199 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy
 - yarn.client.max-cached-nodemanagers-proxies : 0
2019-08-21 20:00:28,200 INFO  org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
 - Starting ZooKeeperLeaderElectionService ZooKeeperLeaderElectionService{leaderPath='/leader/resource_manager_lock'}.
2019-08-21 20:00:28,203 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - No state backend has been configured, using default (Memory / JobManager) MemoryStateBackend
(data in heap memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null',
asynchronous: TRUE, maxStateSize: 5242880)
2019-08-21 20:00:28,211 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - ResourceManager akka.tcp://flink@host53:59609/user/resourcemanager was granted leadership
with fencing token 97537c0d61bbf091e37cb6c45b784568
2019-08-21 20:00:28,214 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager
 - Starting the SlotManager.
2019-08-21 20:00:28,225 INFO  org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore
 - Recovering checkpoints from ZooKeeper.
2019-08-21 20:00:28,229 INFO  org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore
 - Found 0 checkpoints in ZooKeeper.
2019-08-21 20:00:28,229 INFO  org.apache.flink.runtime.checkpoint.ZooKeeperCompletedCheckpointStore
 - Trying to fetch 0 checkpoints from storage.
2019-08-21 20:00:28,233 INFO  org.apache.flink.runtime.leaderelection.ZooKeeperLeaderElectionService
 - Starting ZooKeeperLeaderElectionService ZooKeeperLeaderElectionService{leaderPath='/leader/e5ea83238d721e932b4645589660406c/job_manager_lock'}.
2019-08-21 20:00:28,248 INFO  org.apache.flink.runtime.jobmaster.JobManagerRunner        
  - JobManager runner for job StreamKafkaMain (e5ea83238d721e932b4645589660406c) was granted
leadership with session id 12c96c94-ebbd-4991-a457-6f0320d71bc1 at akka.tcp://flink@host53:59609/user/jobmanager_0.
2019-08-21 20:00:28,260 INFO  org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService
 - Starting ZooKeeperLeaderRetrievalService /leader/resource_manager_lock.
2019-08-21 20:00:28,260 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - Starting execution of job StreamKafkaMain (e5ea83238d721e932b4645589660406c) under job
master id a4576f0320d71bc112c96c94ebbd4991.
2019-08-21 20:00:28,261 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Job StreamKafkaMain (e5ea83238d721e932b4645589660406c) switched from state CREATED to
RUNNING.
2019-08-21 20:00:28,266 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (1/4) (ee39866835f7b7387f518e352787a532) switched from CREATED
to SCHEDULED.
2019-08-21 20:00:28,280 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl   
  - Cannot serve slot request, no ResourceManager connected. Adding as pending request [SlotRequestId{f6123820ffb8b3e906b876e9dea4497b}]
2019-08-21 20:00:28,287 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (2/4) (f1a893fc5e2440331c8e6db1874ad220) switched from CREATED
to SCHEDULED.
2019-08-21 20:00:28,288 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl   
  - Cannot serve slot request, no ResourceManager connected. Adding as pending request [SlotRequestId{fdbb2e71c9bc795452687396abff7bf8}]
2019-08-21 20:00:28,288 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (3/4) (4756a0450881325af5fb396a18dbb8ae) switched from CREATED
to SCHEDULED.
2019-08-21 20:00:28,289 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl   
  - Cannot serve slot request, no ResourceManager connected. Adding as pending request [SlotRequestId{13970fab9312acf9d5063b1d1a33aa0e}]
2019-08-21 20:00:28,289 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (4/4) (78f520582607e26b365fca483fc98d4c) switched from CREATED
to SCHEDULED.
2019-08-21 20:00:28,289 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl   
  - Cannot serve slot request, no ResourceManager connected. Adding as pending request [SlotRequestId{0621470f796aa3ea4291d1b8b423e428}]
2019-08-21 20:00:28,289 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - window: (TumblingGroupWindow('w$, 'rowtime, 60000.millis)), select: (COUNT(*) AS pv, COUNT(DISTINCT
curuserid) AS uv, start('w$) AS w$start, end('w$) AS w$end, rowtime('w$) AS w$rowtime, proctime('w$)
AS w$proctime) -> select: (pv, uv, utc2local(w$end) AS EXPR$2) -> to: Row (1/1) (0071e1c7c4f70646914329f81dcbd349)
switched from CREATED to SCHEDULED.
2019-08-21 20:00:28,291 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (1/4) (0a206444b10553cce42fa34ea0b15497) switched from CREATED
to SCHEDULED.
2019-08-21 20:00:28,291 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (2/4) (a5d1bb759ee6c8001313fe39c7982a8b) switched from CREATED
to SCHEDULED.
2019-08-21 20:00:28,291 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (3/4) (9543bbde6538ea9d407af3012fede5c8) switched from CREATED
to SCHEDULED.
2019-08-21 20:00:28,291 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (4/4) (0e8c4210bd81e581fbf354b86397ce41) switched from CREATED
to SCHEDULED.
2019-08-21 20:00:28,291 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (1/4) (39d437bceb033f5bd37747e9142c7f0e) switched from
CREATED to SCHEDULED.
2019-08-21 20:00:28,291 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (2/4) (aa4f64edc4f10da2e8bdbf6ebf0aeb83) switched from
CREATED to SCHEDULED.
2019-08-21 20:00:28,291 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (3/4) (f109918ce195d1c88c2c63a44934217f) switched from
CREATED to SCHEDULED.
2019-08-21 20:00:28,292 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (4/4) (9aa5bb28f3ba75afc1de47bd9a4e5d01) switched from
CREATED to SCHEDULED.
2019-08-21 20:00:28,295 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - Connecting to ResourceManager akka.tcp://flink@host53:59609/user/resourcemanager(97537c0d61bbf091e37cb6c45b784568)
2019-08-21 20:00:28,300 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - Resolved ResourceManager address, beginning registration
2019-08-21 20:00:28,300 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - Registration at ResourceManager attempt 1 (timeout=100ms)
2019-08-21 20:00:28,304 INFO  org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalService
 - Starting ZooKeeperLeaderRetrievalService /leader/e5ea83238d721e932b4645589660406c/job_manager_lock.
2019-08-21 20:00:28,304 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Registering job manager a4576f0320d71bc112c96c94ebbd4991@akka.tcp://flink@host53:59609/user/jobmanager_0
for job e5ea83238d721e932b4645589660406c.
2019-08-21 20:00:28,310 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Registered job manager a4576f0320d71bc112c96c94ebbd4991@akka.tcp://flink@host53:59609/user/jobmanager_0
for job e5ea83238d721e932b4645589660406c.
2019-08-21 20:00:28,313 INFO  org.apache.flink.runtime.jobmaster.JobMaster               
  - JobManager successfully registered at ResourceManager, leader id: 97537c0d61bbf091e37cb6c45b784568.
2019-08-21 20:00:28,314 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl   
  - Requesting new slot [SlotRequestId{13970fab9312acf9d5063b1d1a33aa0e}] and profile ResourceProfile{cpuCores=-1.0,
heapMemoryInMB=-1, directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource
manager.
2019-08-21 20:00:28,315 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Request slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0,
nativeMemoryInMB=0, networkMemoryInMB=0} for job e5ea83238d721e932b4645589660406c with allocation
id faa6a521758e5be4b3dc1955ca0f51b4.
2019-08-21 20:00:28,316 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl   
  - Requesting new slot [SlotRequestId{fdbb2e71c9bc795452687396abff7bf8}] and profile ResourceProfile{cpuCores=-1.0,
heapMemoryInMB=-1, directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource
manager.
2019-08-21 20:00:28,316 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl   
  - Requesting new slot [SlotRequestId{0621470f796aa3ea4291d1b8b423e428}] and profile ResourceProfile{cpuCores=-1.0,
heapMemoryInMB=-1, directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource
manager.
2019-08-21 20:00:28,316 INFO  org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl   
  - Requesting new slot [SlotRequestId{f6123820ffb8b3e906b876e9dea4497b}] and profile ResourceProfile{cpuCores=-1.0,
heapMemoryInMB=-1, directMemoryInMB=0, nativeMemoryInMB=0, networkMemoryInMB=0} from resource
manager.
2019-08-21 20:00:28,321 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Requesting new TaskExecutor container with resources <memory:8192, vCores:2>. Number
pending requests 1.
2019-08-21 20:00:28,322 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Request slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0,
nativeMemoryInMB=0, networkMemoryInMB=0} for job e5ea83238d721e932b4645589660406c with allocation
id a2b419aea4a373686eeed6c46163a528.
2019-08-21 20:00:28,323 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Request slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0,
nativeMemoryInMB=0, networkMemoryInMB=0} for job e5ea83238d721e932b4645589660406c with allocation
id cbc11737ce6729cfe45f24c43180b5c8.
2019-08-21 20:00:28,323 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Requesting new TaskExecutor container with resources <memory:8192, vCores:2>. Number
pending requests 2.
2019-08-21 20:00:28,323 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Request slot with profile ResourceProfile{cpuCores=-1.0, heapMemoryInMB=-1, directMemoryInMB=0,
nativeMemoryInMB=0, networkMemoryInMB=0} for job e5ea83238d721e932b4645589660406c with allocation
id f6566567c173a55b2bc52b72bcb24ff8.
2019-08-21 20:00:31,597 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator  
  - Checkpoint triggering task Source: Custom Source -> Flat Map -> Filter -> Filter
-> Map -> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
-> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime) (1/4) of job
e5ea83238d721e932b4645589660406c is not in state RUNNING but SCHEDULED instead. Aborting checkpoint.
2019-08-21 20:00:33,749 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl      
  - Received new token for : host53:59938
2019-08-21 20:00:33,749 INFO  org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl      
  - Received new token for : host52:33246
2019-08-21 20:00:33,765 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Received new container: container_1566383236573_0003_01_000002 - Remaining pending container
requests: 2
2019-08-21 20:00:33,765 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Removing container request Capability[<memory:8192, vCores:2>]Priority[1]. Pending
container requests 1.
2019-08-21 20:00:34,009 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Creating container launch context for TaskManagers
2019-08-21 20:00:34,010 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Starting TaskManagers
2019-08-21 20:00:34,021 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy
 - Opening proxy : host53:59938
2019-08-21 20:00:34,066 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Received new container: container_1566383236573_0003_01_000003 - Remaining pending container
requests: 1
2019-08-21 20:00:34,066 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Removing container request Capability[<memory:8192, vCores:2>]Priority[1]. Pending
container requests 0.
2019-08-21 20:00:34,088 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Creating container launch context for TaskManagers
2019-08-21 20:00:34,089 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Starting TaskManagers
2019-08-21 20:00:34,090 INFO  org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy
 - Opening proxy : host52:33246
2019-08-21 20:00:36,597 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator  
  - Checkpoint triggering task Source: Custom Source -> Flat Map -> Filter -> Filter
-> Map -> Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime)
-> select: (rowtime, 0 AS $f1, curuserid) -> time attribute: (rowtime) (1/4) of job
e5ea83238d721e932b4645589660406c is not in state RUNNING but SCHEDULED instead. Aborting checkpoint.
2019-08-21 20:00:37,856 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Registering TaskManager with ResourceID container_1566383236573_0003_01_000002 (akka.tcp://flink@host53:34608/user/taskmanager_0)
at ResourceManager
2019-08-21 20:00:39,573 INFO  org.apache.flink.yarn.YarnResourceManager                  
  - Registering TaskManager with ResourceID container_1566383236573_0003_01_000003 (akka.tcp://flink@host52:32931/user/taskmanager_0)
at ResourceManager
2019-08-21 20:00:39,659 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (1/4) (ee39866835f7b7387f518e352787a532) switched from SCHEDULED
to DEPLOYING.
2019-08-21 20:00:39,659 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Source: Custom Source -> Flat Map -> Filter -> Filter -> Map ->
Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime) -> select: (rowtime,
0 AS $f1, curuserid) -> time attribute: (rowtime) (1/4) (attempt #0) to container_1566383236573_0003_01_000002
@ host53 (dataPort=52456)
2019-08-21 20:00:39,664 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (2/4) (f1a893fc5e2440331c8e6db1874ad220) switched from SCHEDULED
to DEPLOYING.
2019-08-21 20:00:39,664 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Source: Custom Source -> Flat Map -> Filter -> Filter -> Map ->
Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime) -> select: (rowtime,
0 AS $f1, curuserid) -> time attribute: (rowtime) (2/4) (attempt #0) to container_1566383236573_0003_01_000002
@ host53 (dataPort=52456)
2019-08-21 20:00:39,665 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (3/4) (4756a0450881325af5fb396a18dbb8ae) switched from SCHEDULED
to DEPLOYING.
2019-08-21 20:00:39,666 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Source: Custom Source -> Flat Map -> Filter -> Filter -> Map ->
Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime) -> select: (rowtime,
0 AS $f1, curuserid) -> time attribute: (rowtime) (3/4) (attempt #0) to container_1566383236573_0003_01_000003
@ host52 (dataPort=39650)
2019-08-21 20:00:39,666 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (4/4) (78f520582607e26b365fca483fc98d4c) switched from SCHEDULED
to DEPLOYING.
2019-08-21 20:00:39,666 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Source: Custom Source -> Flat Map -> Filter -> Filter -> Map ->
Timestamps/Watermarks -> from: (request, curuserid, timelong, rowtime) -> select: (rowtime,
0 AS $f1, curuserid) -> time attribute: (rowtime) (4/4) (attempt #0) to container_1566383236573_0003_01_000003
@ host52 (dataPort=39650)
2019-08-21 20:00:39,666 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - window: (TumblingGroupWindow('w$, 'rowtime, 60000.millis)), select: (COUNT(*) AS pv, COUNT(DISTINCT
curuserid) AS uv, start('w$) AS w$start, end('w$) AS w$end, rowtime('w$) AS w$rowtime, proctime('w$)
AS w$proctime) -> select: (pv, uv, utc2local(w$end) AS EXPR$2) -> to: Row (1/1) (0071e1c7c4f70646914329f81dcbd349)
switched from SCHEDULED to DEPLOYING.
2019-08-21 20:00:39,666 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying window: (TumblingGroupWindow('w$, 'rowtime, 60000.millis)), select: (COUNT(*)
AS pv, COUNT(DISTINCT curuserid) AS uv, start('w$) AS w$start, end('w$) AS w$end, rowtime('w$)
AS w$rowtime, proctime('w$) AS w$proctime) -> select: (pv, uv, utc2local(w$end) AS EXPR$2)
-> to: Row (1/1) (attempt #0) to container_1566383236573_0003_01_000002 @ host53 (dataPort=52456)
2019-08-21 20:00:39,670 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (1/4) (0a206444b10553cce42fa34ea0b15497) switched from SCHEDULED
to DEPLOYING.
2019-08-21 20:00:39,670 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Sink: Print to Std. Out (1/4) (attempt #0) to container_1566383236573_0003_01_000003
@ host52 (dataPort=39650)
2019-08-21 20:00:39,671 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (2/4) (a5d1bb759ee6c8001313fe39c7982a8b) switched from SCHEDULED
to DEPLOYING.
2019-08-21 20:00:39,672 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Sink: Print to Std. Out (2/4) (attempt #0) to container_1566383236573_0003_01_000003
@ host52 (dataPort=39650)
2019-08-21 20:00:39,672 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (3/4) (9543bbde6538ea9d407af3012fede5c8) switched from SCHEDULED
to DEPLOYING.
2019-08-21 20:00:39,672 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Sink: Print to Std. Out (3/4) (attempt #0) to container_1566383236573_0003_01_000002
@ host53 (dataPort=52456)
2019-08-21 20:00:39,672 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (4/4) (0e8c4210bd81e581fbf354b86397ce41) switched from SCHEDULED
to DEPLOYING.
2019-08-21 20:00:39,673 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Sink: Print to Std. Out (4/4) (attempt #0) to container_1566383236573_0003_01_000002
@ host53 (dataPort=52456)
2019-08-21 20:00:39,673 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (1/4) (39d437bceb033f5bd37747e9142c7f0e) switched from
SCHEDULED to DEPLOYING.
2019-08-21 20:00:39,673 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Map -> Sink: Print to Std. Out (1/4) (attempt #0) to container_1566383236573_0003_01_000003
@ host52 (dataPort=39650)
2019-08-21 20:00:39,675 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (2/4) (aa4f64edc4f10da2e8bdbf6ebf0aeb83) switched from
SCHEDULED to DEPLOYING.
2019-08-21 20:00:39,675 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Map -> Sink: Print to Std. Out (2/4) (attempt #0) to container_1566383236573_0003_01_000003
@ host52 (dataPort=39650)
2019-08-21 20:00:39,675 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (3/4) (f109918ce195d1c88c2c63a44934217f) switched from
SCHEDULED to DEPLOYING.
2019-08-21 20:00:39,675 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Map -> Sink: Print to Std. Out (3/4) (attempt #0) to container_1566383236573_0003_01_000002
@ host53 (dataPort=52456)
2019-08-21 20:00:39,676 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (4/4) (9aa5bb28f3ba75afc1de47bd9a4e5d01) switched from
SCHEDULED to DEPLOYING.
2019-08-21 20:00:39,676 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Deploying Map -> Sink: Print to Std. Out (4/4) (attempt #0) to container_1566383236573_0003_01_000002
@ host53 (dataPort=52456)
2019-08-21 20:00:39,782 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (4/4) (0e8c4210bd81e581fbf354b86397ce41) switched from DEPLOYING
to RUNNING.
2019-08-21 20:00:39,783 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (3/4) (9543bbde6538ea9d407af3012fede5c8) switched from DEPLOYING
to RUNNING.
2019-08-21 20:00:39,784 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (4/4) (9aa5bb28f3ba75afc1de47bd9a4e5d01) switched from
DEPLOYING to RUNNING.
2019-08-21 20:00:39,787 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (3/4) (f109918ce195d1c88c2c63a44934217f) switched from
DEPLOYING to RUNNING.
2019-08-21 20:00:39,806 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (2/4) (f1a893fc5e2440331c8e6db1874ad220) switched from DEPLOYING
to RUNNING.
2019-08-21 20:00:39,809 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (1/4) (ee39866835f7b7387f518e352787a532) switched from DEPLOYING
to RUNNING.
2019-08-21 20:00:39,809 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - window: (TumblingGroupWindow('w$, 'rowtime, 60000.millis)), select: (COUNT(*) AS pv, COUNT(DISTINCT
curuserid) AS uv, start('w$) AS w$start, end('w$) AS w$end, rowtime('w$) AS w$rowtime, proctime('w$)
AS w$proctime) -> select: (pv, uv, utc2local(w$end) AS EXPR$2) -> to: Row (1/1) (0071e1c7c4f70646914329f81dcbd349)
switched from DEPLOYING to RUNNING.
2019-08-21 20:00:39,991 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (1/4) (0a206444b10553cce42fa34ea0b15497) switched from DEPLOYING
to RUNNING.
2019-08-21 20:00:39,992 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Sink: Print to Std. Out (2/4) (a5d1bb759ee6c8001313fe39c7982a8b) switched from DEPLOYING
to RUNNING.
2019-08-21 20:00:39,996 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (1/4) (39d437bceb033f5bd37747e9142c7f0e) switched from
DEPLOYING to RUNNING.
2019-08-21 20:00:39,997 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Map -> Sink: Print to Std. Out (2/4) (aa4f64edc4f10da2e8bdbf6ebf0aeb83) switched from
DEPLOYING to RUNNING.
2019-08-21 20:00:40,005 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (4/4) (78f520582607e26b365fca483fc98d4c) switched from DEPLOYING
to RUNNING.
2019-08-21 20:00:40,006 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph     
  - Source: Custom Source -> Flat Map -> Filter -> Filter -> Map -> Timestamps/Watermarks
-> from: (request, curuserid, timelong, rowtime) -> select: (rowtime, 0 AS $f1, curuserid)
-> time attribute: (rowtime) (3/4) (4756a0450881325af5fb396a18dbb8ae) switched from DEPLOYING
to RUNNING.
2019-08-21 20:00:41,605 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator  
  - Triggering checkpoint 1 @ 1566388841597 for job e5ea83238d721e932b4645589660406c.
2019-08-21 20:00:42,466 INFO  org.apache.flink.runtime.checkpoint.CheckpointCoordinator  
  - Completed checkpoint 1 for job e5ea83238d721e932b4645589660406c (7994 bytes in 821 ms).



------------------ 原始邮件 ------------------
发件人: "Yun Tang"<myasuka@live.com>;
发送时间: 2019年8月22日(星期四) 中午11:08
收件人: "user-zh"<user-zh@flink.apache.org>;

主题: Re: flink启动等待10分钟问题



Hi

Flink on YARN作业启动时间长,有很多原因,例如资源不够在等待,container申请的时候又退出了。默认的slot
request的timeout时间是5min,感觉你的作业应该是可能遇到了一个slot request
timeout,然后又重新申请。最好能提供一下jobmanager的日志才好进一步分析。

祝好
唐云
________________________________
From: 々守护々 <346531110@qq.com>
Sent: Thursday, August 22, 2019 11:04
To: user-zh <user-zh@flink.apache.org>
Subject: flink启动等待10分钟问题

各位Flink社区大佬,您好!
      我使用Flink SQL (Flink 1.8.1),在hadoop 集群中调整了一下yarn-site.xml文件的内存参数:
        <property>
                <name>yarn.nodemanager.resource.memory-mb</name>
                <value>16384</value>
        </property>
        <property>
                <name>yarn.scheduler.minimum-allocation-mb</name>
                <value>1024</value>
        </property>
        <property>
                <name>yarn.scheduler.maximum-allocation-mb</name>
                <value>8192</value>
        </property>



然后我去启动flink on yarn任务,启动参数:./flink-1.8.1/bin/flink run -m yarn-cluster
-p 4 -d -c streaming.StreamKafkaMain -yn 4 -ys 2 -yjm 2048 -ytm 8192  -ynm bigdata6 ./flink_bigdata6.jar
最终能能启动成功,就是启动要等待10分钟,这个等待时间太长了,请问各位大佬有什么解决的方法吗?
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message