tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-1647) Issue with caching of events in VertexManager::onRootVertexInitialized
Date Wed, 08 Oct 2014 23:59:33 GMT

    [ https://issues.apache.org/jira/browse/TEZ-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164416#comment-14164416

Jeff Zhang commented on TEZ-1647:

Looks like we should use List instead of Map for cachedInputEvents
ConcurrentHashMap<String, List<TezEvent>> cachedRootInputEventMap;

> Issue with caching of events in VertexManager::onRootVertexInitialized 
> -----------------------------------------------------------------------
>                 Key: TEZ-1647
>                 URL: https://issues.apache.org/jira/browse/TEZ-1647
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Hitesh Shah
>            Assignee: Jeff Zhang
> Came across a use-case in Hive where the current functionality does not work. 
> Consider a vertex with 2 inputs ( i1, i2 ):
> The custom vertex manager in Hive generates events only when both i1 and i2 are initialized.

> Therefore vertexManagerPlugin::onRootInitialized(i1) does nothing but for vertexManagerPLugin::onRootIniitialized(i2)
, it calls context.addEvents(i1) and context.addEvents(i2). When this happens, the events
generated for i1 never get sent to the VertexImpl to be routed to the tasks. 
> What this implies is that

This message was sent by Atlassian JIRA

View raw message