flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Crane <jeffcra...@yahoo.com>
Subject Re: Install 1.7.2 on EC2 - No task slots - 2019
Date Tue, 26 Mar 2019 17:40:08 GMT
  There are 2 out files and a log file. I will describe the steps to take below, but I have
not found anything to indicate there's a problem when it cannot allocate any resources (but
otherwise runs).
----------start default free tier aws instanceedit security group to allow 8081 incomingsudo
yum install java# wget http://ftp.naz.com/apache/flink/flink-1.7.2/flink-1.7.2-bin-scala_2.12.tgz#
tar xzvf flink-1.7.2-bin-scala_2.12.tgz# cd flink-1.7.2/# ./bin/start-cluster.shWeb interface
shows 0,0,0 for resources.While there are 2 other .out files, I dunno if those are important.Here
is the log of the only .log -flink-ec2-user-standalonesession-0-ip-10-0-0-204.us-west-1.compute.internal.log

2019-03-26 15:43:12,488 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint   
     - --------------------------------------------------------------------------------2019-03-26
15:43:12,489 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - 
Starting StandaloneSessionClusterEntrypoint (Version: 1.7.2, Rev:ceba8af, Date:11.02.2019
@ 14:17:09 UTC)2019-03-26 15:43:12,489 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -  OS current user: ec2-user2019-03-26 15:43:12,489 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -  Current Hadoop/Kerberos user: <no hadoop dependency found>2019-03-26
15:43:12,489 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - 
JVM: OpenJDK 64-Bit Server VM - Oracle Corporation - 1.8/25.191-b122019-03-26 15:43:12,490
INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         -  Maximum heap
size: 989 MiBytes2019-03-26 15:43:12,490 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -  JAVA_HOME: (not set)2019-03-26 15:43:12,490 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -  No Hadoop Dependency available2019-03-26 15:43:12,490 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -  JVM Options:2019-03-26 15:43:12,490 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -     -Xms1024m2019-03-26 15:43:12,491 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -     -Xmx1024m2019-03-26 15:43:12,491 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -     -Dlog.file=/home/ec2-user/flink-1.7.2/log/flink-ec2-user-standalonesession-0-ip-10-0-0-204.us-west-1.compute.internal.log2019-03-26
15:43:12,491 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - 
   -Dlog4j.configuration=file:/home/ec2-user/flink-1.7.2/conf/log4j.properties2019-03-26
15:43:12,491 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - 
   -Dlogback.configurationFile=file:/home/ec2-user/flink-1.7.2/conf/logback.xml2019-03-26
15:43:12,491 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - 
Program Arguments:2019-03-26 15:43:12,491 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -     --configDir2019-03-26 15:43:12,492 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -     /home/ec2-user/flink-1.7.2/conf2019-03-26 15:43:12,492 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -     --executionMode2019-03-26 15:43:12,492 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -     cluster2019-03-26 15:43:12,492 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       -  Classpath: /home/ec2-user/flink-1.7.2/lib/flink-python_2.12-1.7.2.jar:/home/ec2-user/flink-1.7.2/lib/log4j-1.2.17.jar:/home/ec2-user/flink-1.7.2/lib/slf4j-log4j12-1.7.15.jar:/home/ec2-user/flink-1.7.2/lib/flink-dist_2.12-1.7.2.jar:::2019-03-26
15:43:12,492 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - --------------------------------------------------------------------------------2019-03-26
15:43:12,495 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Registered
UNIX signal handlers for [TERM, HUP, INT]2019-03-26 15:43:12,531 INFO  org.apache.flink.configuration.GlobalConfiguration 
          - Loading configuration property: jobmanager.rpc.address, localhost2019-03-26
15:43:12,533 INFO  org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: jobmanager.rpc.port, 61232019-03-26 15:43:12,533 INFO  org.apache.flink.configuration.GlobalConfiguration 
          - Loading configuration property: jobmanager.heap.size, 1024m2019-03-26 15:43:12,534
INFO  org.apache.flink.configuration.GlobalConfiguration            - Loading configuration
property: taskmanager.heap.size, 1024m2019-03-26 15:43:12,534 INFO  org.apache.flink.configuration.GlobalConfiguration 
          - Loading configuration property: taskmanager.numberOfTaskSlots, 12019-03-26
15:43:12,534 INFO  org.apache.flink.configuration.GlobalConfiguration            -
Loading configuration property: parallelism.default, 12019-03-26 15:43:12,535 INFO  org.apache.flink.configuration.GlobalConfiguration 
          - Loading configuration property: rest.port, 80812019-03-26 15:43:12,705 INFO 
org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Starting StandaloneSessionClusterEntrypoint.2019-03-26
15:43:12,705 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Install
default filesystem.2019-03-26 15:43:12,714 INFO  org.apache.flink.core.fs.FileSystem   
                       - Hadoop is not in the classpath/dependencies. The extended
set of supported File Systems via Hadoop is not available.2019-03-26 15:43:12,730 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       - Install security context.2019-03-26 15:43:12,742 INFO  org.apache.flink.runtime.security.modules.HadoopModuleFactory 
- Cannot create Hadoop Security Module because Hadoop cannot be found in the Classpath.2019-03-26
15:43:12,761 INFO  org.apache.flink.runtime.security.SecurityUtils               -
Cannot install HadoopSecurityContext because Hadoop cannot be found in the Classpath.2019-03-26
15:43:12,762 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Initializing
cluster services.2019-03-26 15:43:13,432 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils 
       - Trying to start actor system at localhost:61232019-03-26 15:43:14,281 INFO 
akka.event.slf4j.Slf4jLogger                                  - Slf4jLogger
started2019-03-26 15:43:14,418 INFO  akka.remote.Remoting                     
                    - Starting remoting2019-03-26 15:43:14,669 INFO  akka.remote.Remoting 
                                        - Remoting started; listening
on addresses :[akka.tcp://flink@localhost:6123]2019-03-26 15:43:14,692 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils 
       - Actor system started at akka.tcp://flink@localhost:61232019-03-26 15:43:14,711
WARN  org.apache.flink.configuration.Configuration                  - Config uses
deprecated configuration key 'jobmanager.rpc.address' instead of proper key 'rest.address'2019-03-26
15:43:14,724 INFO  org.apache.flink.runtime.blob.BlobServer                   
  - Created BLOB server storage directory /tmp/blobStore-cbac7ece-9cb3-4bf8-a225-26fe4136067d2019-03-26
15:43:14,728 INFO  org.apache.flink.runtime.blob.BlobServer                   
  - Started BLOB server at 0.0.0.0:45861 - max concurrent requests: 50 - max backlog: 10002019-03-26
15:43:14,760 INFO  org.apache.flink.runtime.metrics.MetricRegistryImpl           -
No metrics reporter configured, no metrics will be exposed/reported.2019-03-26 15:43:14,762
INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint         - Trying to start
actor system at localhost:02019-03-26 15:43:14,813 INFO  akka.event.slf4j.Slf4jLogger   
                              - Slf4jLogger started2019-03-26 15:43:14,829
INFO  akka.remote.Remoting                                         
- Starting remoting2019-03-26 15:43:14,849 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint 
       - Actor system started at akka.tcp://flink-metrics@localhost:381812019-03-26 15:43:14,853
INFO  akka.remote.Remoting                                         
- Remoting started; listening on addresses :[akka.tcp://flink-metrics@localhost:38181]2019-03-26
15:43:14,861 INFO  org.apache.flink.runtime.dispatcher.FileArchivedExecutionGraphStore 
- Initializing FileArchivedExecutionGraphStore: Storage directory /tmp/executionGraphStore-99cbce6e-06d9-4159-b68e-e461b31e41cf,
expiration time 3600000, maximum cache size 52428800 bytes.2019-03-26 15:43:14,898 INFO 
org.apache.flink.runtime.blob.TransientBlobCache              - Created BLOB cache
storage directory /tmp/blobStore-d8d4eb58-74fd-4f6f-9b9b-e71a492286522019-03-26 15:43:14,915
WARN  org.apache.flink.configuration.Configuration                  - Config uses
deprecated configuration key 'jobmanager.rpc.address' instead of proper key 'rest.address'2019-03-26
15:43:14,917 WARN  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Upload
directory /tmp/flink-web-027518bf-4744-4f65-9bc0-c0d8e23853b0/flink-web-upload does not exist,
or has been deleted externally. Previously uploaded files are no longer available.2019-03-26
15:43:14,918 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Created
directory /tmp/flink-web-027518bf-4744-4f65-9bc0-c0d8e23853b0/flink-web-upload for file uploads.2019-03-26
15:43:14,926 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Starting
rest endpoint.2019-03-26 15:43:15,286 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils 
         - Determined location of main cluster component log file: /home/ec2-user/flink-1.7.2/log/flink-ec2-user-standalonesession-0-ip-10-0-0-204.us-west-1.compute.internal.log2019-03-26
15:43:15,286 INFO  org.apache.flink.runtime.webmonitor.WebMonitorUtils           -
Determined location of main cluster component stdout file: /home/ec2-user/flink-1.7.2/log/flink-ec2-user-standalonesession-0-ip-10-0-0-204.us-west-1.compute.internal.out2019-03-26
15:43:15,504 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest
endpoint listening at localhost:80812019-03-26 15:43:15,504 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint 
  - http://localhost:8081 was granted leadership with leaderSessionID=00000000-0000-0000-0000-0000000000002019-03-26
15:43:15,504 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Web
frontend listening at http://localhost:8081.2019-03-26 15:43:15,670 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService 
            - Starting RPC endpoint for org.apache.flink.runtime.resourcemanager.StandaloneResourceManager
at akka://flink/user/resourcemanager .2019-03-26 15:43:15,713 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService 
            - Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher
at akka://flink/user/dispatcher .2019-03-26 15:43:15,744 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager 
- ResourceManager akka.tcp://flink@localhost:6123/user/resourcemanager was granted leadership
with fencing token 000000000000000000000000000000002019-03-26 15:43:15,745 INFO  org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager 
- Starting the SlotManager.2019-03-26 15:43:15,781 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher 
    - Dispatcher akka.tcp://flink@localhost:6123/user/dispatcher was granted leadership
with fencing token 00000000-0000-0000-0000-0000000000002019-03-26 15:43:15,783 INFO  org.apache.flink.runtime.dispatcher.StandaloneDispatcher 
    - Recovering all persisted jobs.

----------On Monday, March 25, 2019, 1:23:45 AM PDT, Konstantin Knauf <konstantin@ververica.com>
wrote:

Hi Jeff, 

do you see any log files in the log directory of your Flink installation directory? If so,
please share them. 

Cheers, 

Konstantin  
Mime
View raw message