Return-Path: X-Original-To: apmail-tajo-dev-archive@minotaur.apache.org Delivered-To: apmail-tajo-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 22476102F3 for ; Fri, 25 Oct 2013 03:54:01 +0000 (UTC) Received: (qmail 35730 invoked by uid 500); 25 Oct 2013 03:54:01 -0000 Delivered-To: apmail-tajo-dev-archive@tajo.apache.org Received: (qmail 35631 invoked by uid 500); 25 Oct 2013 03:53:57 -0000 Mailing-List: contact dev-help@tajo.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tajo.incubator.apache.org Delivered-To: mailing list dev@tajo.incubator.apache.org Received: (qmail 35614 invoked by uid 99); 25 Oct 2013 03:53:53 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 25 Oct 2013 03:53:53 +0000 X-ASF-Spam-Status: No, hits=-2000.4 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Fri, 25 Oct 2013 03:53:52 +0000 Received: (qmail 35586 invoked by uid 99); 25 Oct 2013 03:53:30 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 25 Oct 2013 03:53:30 +0000 Date: Fri, 25 Oct 2013 03:53:30 +0000 (UTC) From: "Jihoon Son (JIRA)" To: dev@tajo.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (TAJO-275) Separating QueryMaster and TaskRunner roles in worker MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/TAJO-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804986#comment-13804986 ] Jihoon Son commented on TAJO-275: --------------------------------- If this is a temporal solution to separate QueryMaster and TaskRunner, it's ok. But, I got a following message during applying the patch. It seems to require a rebase. {code} $ patch -p0 < TAJO-275.patch patching file pom.xml patching file tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java patching file tajo-core/tajo-core-backend/pom.xml patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoContainerProxy.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterService.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryJobManager.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMasterManagerService.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java Hunk #11 FAILED at 346. Hunk #12 succeeded at 375 (offset 1 line). Hunk #13 succeeded at 404 (offset 1 line). 1 out of 13 hunks FAILED -- saving rejects to file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java.rej patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/WorkerResource.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/WorkerResourceManager.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/YarnTajoResourceManager.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorker.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerManagerService.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/Task.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TaskRunner.java patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TaskRunnerManager.java patching file tajo-core/tajo-core-backend/src/main/proto/QueryMasterProtocol.proto patching file tajo-core/tajo-core-backend/src/main/proto/TajoMasterProtocol.proto patching file tajo-core/tajo-core-backend/src/main/proto/TajoWorkerProtocol.proto patching file tajo-core/tajo-core-backend/src/main/resources/tajo-default.xml patching file tajo-core/tajo-core-backend/src/main/resources/webapps/admin/cluster.jsp Hunk #1 succeeded at 34 (offset 19 lines). Hunk #2 succeeded at 84 (offset 19 lines). patching file tajo-core/tajo-core-backend/src/main/resources/webapps/admin/index.jsp Hunk #1 succeeded at 20 with fuzz 1 (offset 19 lines). Hunk #2 succeeded at 134 (offset 19 lines). Hunk #3 succeeded at 142 (offset 19 lines). patching file tajo-core/tajo-core-backend/src/main/resources/webapps/admin/query.jsp Hunk #1 succeeded at 20 with fuzz 1 (offset 19 lines). Hunk #2 succeeded at 37 (offset 19 lines). Hunk #3 succeeded at 73 (offset 19 lines). Hunk #4 succeeded at 105 (offset 19 lines). patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/env.jsp Hunk #1 succeeded at 39 (offset 19 lines). patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/index.jsp Hunk #1 succeeded at 29 (offset 19 lines). Hunk #2 succeeded at 55 (offset 19 lines). Hunk #3 succeeded at 82 (offset 19 lines). Hunk #4 succeeded at 97 (offset 19 lines). Hunk #5 succeeded at 108 (offset 19 lines). Hunk #6 succeeded at 134 (offset 19 lines). patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/querydetail.jsp Hunk #1 succeeded at 32 (offset 19 lines). patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/queryplan.jsp Hunk #1 succeeded at 39 (offset 19 lines). patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/querytasks.jsp Hunk #1 succeeded at 46 (offset 19 lines). patching file tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/TajoTestingCluster.java patching file tajo-dist/src/main/bin/start-tajo.sh patching file tajo-dist/src/main/bin/stop-tajo.sh patching file tajo-dist/src/main/bin/tajo patching file tajo-dist/src/main/bin/tajo-config.sh patching file tajo-dist/src/main/conf/tajo-env.sh {code} > Separating QueryMaster and TaskRunner roles in worker > ----------------------------------------------------- > > Key: TAJO-275 > URL: https://issues.apache.org/jira/browse/TAJO-275 > Project: Tajo > Issue Type: Improvement > Reporter: Keuntae Park > Assignee: Keuntae Park > Attachments: TAJO-275.patch > > > In C++ implementation of Tajo worker, > I think it is better to maintain Java code of QueryMaster and implement only TaskRunner as C++ code, rather than implementing both QueryMaster and TaskRunner as C++ code. > Accordingly, standby mode worker will have following 3 modes: > 1) TaskRunner + QueryMaster (current implementation) > 2) TaskRunner only (C++) > 3) QueryMaster only (java) > And, most workers work as C++ TaskRunner, and 1 or 2 workers work as separate QueryMasters (java process). > For backward compatibility, default mode should be mode 1 (TaskRunner + QueryMaster) > To achieve the goal, it needs to separate java TaskRunner and java QueryMaster Worker first. > (And then, implement C++ TaskRunner) -- This message was sent by Atlassian JIRA (v6.1#6144)