tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From László Bodor (Jira) <j...@apache.org>
Subject [jira] [Updated] (TEZ-4236) DAGClientServer is not really needed to be started/used in local mode
Date Thu, 28 Jan 2021 21:25:00 GMT

     [ https://issues.apache.org/jira/browse/TEZ-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

László Bodor updated TEZ-4236:
------------------------------
    Fix Version/s: 0.10.1

> DAGClientServer is not really needed to be started/used in local mode
> ---------------------------------------------------------------------
>
>                 Key: TEZ-4236
>                 URL: https://issues.apache.org/jira/browse/TEZ-4236
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: László Bodor
>            Assignee: László Bodor
>            Priority: Major
>             Fix For: 0.10.1
>
>         Attachments: TEZ-4236.01.patch, TEZ-4236.02.patch, TEZ-4236.03.patch, TEZ-4236.04.patch,
TEZ-4236.05.patch, TEZ-4236.06.patch
>
>
> For a customer, we found that they try to run hive on tez local mode tests, and due to
strict firewall rules and security policy, the DagClientServer struggles to start or respond
to connections (think about corporate laptops here). Instead of investigating the network-related
workarounds, let's find another way and decide if DAGAppMaster really needs this server. 
> LocalClient has an embedded [DAGAppMaster|https://github.com/apache/tez/blob/master/tez-dag/src/main/java/org/apache/tez/client/LocalClient.java#L65],
so at the first sight, it doesn't make sense to rely on the network environment in terms of
AM usage. Hopefully, AM can become more lightweight in the long term if we try to eliminate
pointless network usage. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message