beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Work logged] (BEAM-4141) Data channel deadlocks when user function fails
Date Thu, 19 Apr 2018 16:38:00 GMT


ASF GitHub Bot logged work on BEAM-4141:

                Author: ASF GitHub Bot
            Created on: 19/Apr/18 16:37
            Start Date: 19/Apr/18 16:37
    Worklog Time Spent: 10m 
      Work Description: wcn3 opened a new pull request #5184: BEAM-4141: Drain source when
user function processing fails.
   When the user function fails, the data is continued to be buffered up by
   the data plane. Since nothing is consuming it, the data plane will
   eventually block and no other data channels can be serviced, causing the
   entire worker to essentially be deadlocked.
   Also added a minor cleanup to refer to the self data channel as c instead of m.

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

Issue Time Tracking

            Worklog Id:     (was: 92672)
            Time Spent: 10m
    Remaining Estimate: 0h

> Data channel deadlocks when user function fails
> -----------------------------------------------
>                 Key: BEAM-4141
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-go
>    Affects Versions: Not applicable
>            Reporter: Bill Neubauer
>            Assignee: Bill Neubauer
>            Priority: Major
>             Fix For: Not applicable
>          Time Spent: 10m
>  Remaining Estimate: 0h
> There is a deadlock condition in the data channel code that occurs when a user function
fails while processing an element. The producer for the data channel is continuing to send
information across a channel, but the intended consumer has stopped listening. Unfortunately,
this channel blocks the entire data channel, blocking data for any other DoFn that might be
running, causing the whole worker to deadlock.

This message was sent by Atlassian JIRA

View raw message