tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-1703) Exception handling for InputInitializer
Date Fri, 31 Oct 2014 05:41:33 GMT

    [ https://issues.apache.org/jira/browse/TEZ-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191404#comment-14191404

Jeff Zhang commented on TEZ-1703:

[~sseth] The origin patch has one race issue in test case. Previously I'd like simulate the
behavior of  INIT_FAILED after INIT_SUCCEEDED in InputInitliazed, INIT_SUCCEEDED will cause
the RootInputInitializerManager shutdown, so I make the InputIntializer thread sleep 1 second
to wait for the RootInputInitializerManager shutdown and catch the exception to throw it.
But the issue here is that executor.shutdownNow() is not blocking method, so here would result
in a race issue between InputInitializer thread and AsyncDispatcher thread.  

It's not easy to simulate the behavior of INIT_FAILED after INIT_SUCCEEDED in InputInitializer
, so in the new patch I did it in AsyncDispatcher thread.

commit 8f8a81f7a17f9018ae4e87bf0fca9d6cdc0a5ba4 (HEAD, origin/master, origin/HEAD, master,
Author: Jeff Zhang <zjffdu@apache.org>
Date:   Fri Oct 31 13:30:10 2014 +0800

    TEZ-1703. addendum - fix flaky test. (zjffdu)

> Exception handling for InputInitializer
> ---------------------------------------
>                 Key: TEZ-1703
>                 URL: https://issues.apache.org/jira/browse/TEZ-1703
>             Project: Apache Tez
>          Issue Type: Bug
>    Affects Versions: 0.5.1
>            Reporter: Jeff Zhang
>            Assignee: Jeff Zhang
>             Fix For: 0.5.2
>         Attachments: TEZ-1703-2.patch, TEZ-1703-3.patch, TEZ-1703-4.patch, TEZ-1703.patch
> For handleInputInitializerEvent - this should be fairly straightfoward to handle. At
the moment this is an inline call from within the AsyncDispatcher, and will end up causing
a RuntimeException. The RuntimeException can be changed to a AMUserCodeException which will
take care of this.
> For onVertexStateUpdated, this eventually gets invoked from within RootInputInitializerManager.
Catching exceptions there and sending a RootInputInitialzierFailedEvent should be enough to
fix this ? May require some state machine changes to handle this event on a few more states.

This message was sent by Atlassian JIRA

View raw message