beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-5529) Dataflow runner raises AssertionError if job takes > 50 seconds to go from PENDING to RUNNING
Date Wed, 03 Oct 2018 22:25:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-5529?focusedWorklogId=150956&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-150956
]

ASF GitHub Bot logged work on BEAM-5529:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 03/Oct/18 22:24
            Start Date: 03/Oct/18 22:24
    Worklog Time Spent: 10m 
      Work Description: aaltay commented on issue #6535: [BEAM-5529] Before starting the final
countdown, check that job is not in pre-running state
URL: https://github.com/apache/beam/pull/6535#issuecomment-426824008
 
 
   Run Python PostCommit

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 150956)
    Time Spent: 1h 50m  (was: 1h 40m)

> Dataflow runner raises AssertionError if job takes > 50 seconds to go from PENDING
to RUNNING
> ---------------------------------------------------------------------------------------------
>
>                 Key: BEAM-5529
>                 URL: https://issues.apache.org/jira/browse/BEAM-5529
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-py-core
>    Affects Versions: 2.6.0
>            Reporter: Joar Wandborg
>            Assignee: Ahmet Altay
>            Priority: Major
>          Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> To reproduce:
>  - Try to run a Pipeline that takes longer than 50 seconds to go from JOB_STATE_PENDING
to JOB_STATE_RUNNING using "with Pipeline() as pipeline:" (automatically executes {{pipeline.run().wait_until_finish()}}
via {{Pipeline.__exit__}}) on Dataflow.
>  - Your script will raise "AssertionError: Job did not reach to a terminal state after
waiting indefinitely." (from [https://github.com/apache/beam/blob/766a1dc5b6523a0b7a3ce0a9ab78901cf17c9013/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L1152-L1153])
> I think the issue is that the block at [https://github.com/apache/beam/blob/766a1dc5b6523a0b7a3ce0a9ab78901cf17c9013/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py#L147-L160]
unintentionally exits if it takes > 50 seconds for a job to go from JOB_STATE_PENDING to
JOB_STATE_RUNNING.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message