From issues-return-43624-archive-asf-public=cust-asf.ponee.io@tez.apache.org Thu Jan 28 10:39:02 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 492D7180660 for ; Thu, 28 Jan 2021 11:39:02 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 8313543B3A for ; Thu, 28 Jan 2021 10:39:01 +0000 (UTC) Received: (qmail 35349 invoked by uid 500); 28 Jan 2021 10:39:01 -0000 Mailing-List: contact issues-help@tez.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tez.apache.org Delivered-To: mailing list issues@tez.apache.org Received: (qmail 35340 invoked by uid 99); 28 Jan 2021 10:39:01 -0000 Received: from ec2-52-204-25-47.compute-1.amazonaws.com (HELO mailrelay1-ec2-va.apache.org) (52.204.25.47) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jan 2021 10:39:01 +0000 Received: from jira2-he-de.apache.org (jira2-he-de.apache.org [168.119.33.54]) by mailrelay1-ec2-va.apache.org (ASF Mail Server at mailrelay1-ec2-va.apache.org) with ESMTPS id 24AFE3E978 for ; Thu, 28 Jan 2021 10:39:01 +0000 (UTC) Received: from jira2-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira2-he-de.apache.org (ASF Mail Server at jira2-he-de.apache.org) with ESMTP id 6C2BCC801E1 for ; Thu, 28 Jan 2021 10:39:00 +0000 (UTC) Date: Thu, 28 Jan 2021 10:39:00 +0000 (UTC) From: "Hadoop QA (Jira)" To: issues@tez.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (TEZ-4236) Investigate if DAGAppMaster's DAGClientServer is really need to be started/used in local mode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/TEZ-4236?page=3Dcom.atlassian.j= ira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D172734= 90#comment-17273490 ]=20 Hadoop QA commented on TEZ-4236: -------------------------------- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 31s= {color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0= m 1s{color} | {color:green} No case conflicting files found. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0= m 0s{color} | {color:green} The patch does not contain any @author tags. {= color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green}= 0m 0s{color} | {color:green} The patch appears to include 3 new or modif= ied test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s= {color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}= 4m 17s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1= m 28s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}= 1m 22s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1= m 32s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m 4= 1s{color} | {color:blue} Used deprecated FindBugs config; considering switc= hing to SpotBugs. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} = 2m 42s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s= {color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}= 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0= m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m = 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}= 0m 14s{color} | {color:green} tez-api: The patch generated 0 new + 154 un= changed - 2 fixed =3D 154 total (was 156) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:oran= ge} 0m 18s{color} | {color:orange} tez-dag: The patch generated 1 new + 11= 3 unchanged - 0 fixed =3D 114 total (was 113) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}= 0m 10s{color} | {color:green} The patch passed checkstyle in tez-tests {c= olor} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green}= 0m 0s{color} | {color:green} The patch has no whitespace issues. {color}= | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0= m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} = 2m 30s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 4= 7s{color} | {color:green} tez-api in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m 1= 2s{color} | {color:green} tez-dag in the patch passed. {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 44m 1= 6s{color} | {color:green} tez-tests in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green}= 0m 30s{color} | {color:green} The patch does not generate ASF License war= nings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 70m 22s{colo= r} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | ClientAPI=3D1.41 ServerAPI=3D1.41 base: https://ci-hadoop.apache= .org/job/PreCommit-TEZ-Build/98/artifact/out/Dockerfile | | JIRA Issue | TEZ-4236 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/1301955= 5/TEZ-4236.05.patch | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs = checkstyle compile | | uname | Linux 5dd89e42fb84 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 2= 3:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/tez.sh | | git revision | master / 033ba9160 | | Default Java | Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01 | | checkstyle | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/98/arti= fact/out/diff-checkstyle-tez-dag.txt | | Test Results | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/98/t= estReport/ | | Max. process+thread count | 1328 (vs. ulimit of 5500) | | modules | C: tez-api tez-dag tez-tests U: . | | Console output | https://ci-hadoop.apache.org/job/PreCommit-TEZ-Build/98/= console | | versions | git=3D2.17.1 maven=3D3.6.0 findbugs=3D3.0.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. > Investigate if DAGAppMaster's DAGClientServer is really need to be starte= d/used in local mode > -------------------------------------------------------------------------= -------------------- > > Key: TEZ-4236 > URL: https://issues.apache.org/jira/browse/TEZ-4236 > Project: Apache Tez > Issue Type: Improvement > Reporter: L=C3=A1szl=C3=B3 Bodor > Assignee: L=C3=A1szl=C3=B3 Bodor > Priority: Major > Attachments: TEZ-4236.01.patch, TEZ-4236.02.patch, TEZ-4236.03.pa= tch, TEZ-4236.04.patch, TEZ-4236.05.patch > > > For a customer, we found that they try to run hive on tez local mode test= s, and due to strict firewall rules and security policy, the DagClientServe= r struggles to start or respond to connections (think about corporate lapto= ps here). Instead of investigating the network-related workarounds, let's f= ind another way and decide if DAGAppMaster really needs this server.=20 > LocalClient has an embedded [DAGAppMaster|https://github.com/apache/tez/b= lob/master/tez-dag/src/main/java/org/apache/tez/client/LocalClient.java#L65= ], so at the first sight, it doesn't make sense to rely on the network envi= ronment in terms of AM usage. Hopefully, AM can become more lightweight in = the long term if we try to eliminate pointless network usage.=20 -- This message was sent by Atlassian Jira (v8.3.4#803005)