flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5183) [py] Support multiple jobs per Python plan file
Date Thu, 09 Feb 2017 17:01:42 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859800#comment-15859800

ASF GitHub Bot commented on FLINK-5183:

Github user GEOFBOT commented on the issue:

    It may have worked with a smaller file, but there may be issues with heavier jobs. When
I ran a more computationally intensive and time consuming job, the first job of the Python
file ran successfully. The second job of the file was then submitted:
    02/09/2017 16:39:43	DataSink (CsvSink)(4/5) switched to FINISHED 
    02/09/2017 16:39:43	Job execution switched to status FINISHED.
    2017-02-09 16:40:26,470 INFO  org.apache.flink.yarn.YarnClusterClient                
      - Waiting until all TaskManagers have connected
    Waiting until all TaskManagers have connected
    2017-02-09 16:40:26,476 INFO  org.apache.flink.yarn.YarnClusterClient                
      - TaskManager status (5/5)
    TaskManager status (5/5)
    2017-02-09 16:40:26,476 INFO  org.apache.flink.yarn.YarnClusterClient                
      - All TaskManagers are connected
    All TaskManagers are connected
    2017-02-09 16:40:26,480 INFO  org.apache.flink.yarn.YarnClusterClient                
      - Submitting job with JobID: b226f5f18a78bc386bd1b1b6d30515ea. Waiting for job completion.
    Submitting job with JobID: b226f5f18a78bc386bd1b1b6d30515ea. Waiting for job completion.
    Connected to JobManager at Actor[akka.tcp://flink@<snip>.ec2.internal:35598/user/jobmanager#68430682]
    However, Flink does not receive or respond to this new job. Instead, the client terminates
with a timeout error:
    Caused by: org.apache.flink.runtime.client.JobClientActorSubmissionTimeoutException: Job
submission to the JobManager timed out. You may increase 'akka.client.timeout' in case the
JobManager needs more time to configure and confirm the job submission.
    	at org.apache.flink.runtime.client.JobSubmissionClientActor.handleCustomMessage(JobSubmissionClientActor.java:119)
    	at org.apache.flink.runtime.client.JobClientActor.handleMessage(JobClientActor.java:239)
    	at org.apache.flink.runtime.akka.FlinkUntypedActor.handleLeaderSessionID(FlinkUntypedActor.java:88)
    	at org.apache.flink.runtime.akka.FlinkUntypedActor.onReceive(FlinkUntypedActor.java:68)
    	at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)
    I tried setting `akka.client.timeout` to 20 minutes, but Flink is still not receiving
the second job. I suspect this may be an issue with this patch.

> [py] Support multiple jobs per Python plan file
> -----------------------------------------------
>                 Key: FLINK-5183
>                 URL: https://issues.apache.org/jira/browse/FLINK-5183
>             Project: Flink
>          Issue Type: Improvement
>          Components: Python API
>    Affects Versions: 1.1.3
>            Reporter: Geoffrey Mon
>            Priority: Minor
> Support running multiple jobs per Python plan file.

This message was sent by Atlassian JIRA

View raw message