tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jihoon Son (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TAJO-275) Separating QueryMaster and TaskRunner roles in worker
Date Fri, 25 Oct 2013 03:53:30 GMT

    [ https://issues.apache.org/jira/browse/TAJO-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13804986#comment-13804986
] 

Jihoon Son commented on TAJO-275:
---------------------------------

If this is a temporal solution to separate QueryMaster and TaskRunner, it's ok.
But, I got a following message during applying the patch.
It seems to require a rebase.
{code}
$ patch -p0 < TAJO-275.patch 
patching file pom.xml
patching file tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
patching file tajo-core/tajo-core-backend/pom.xml
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoContainerProxy.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterService.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryInProgress.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryInfo.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryJobManager.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMaster.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/QueryMasterManagerService.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java
Hunk #11 FAILED at 346.
Hunk #12 succeeded at 375 (offset 1 line).
Hunk #13 succeeded at 404 (offset 1 line).
1 out of 13 hunks FAILED -- saving rejects to file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/TajoWorkerResourceManager.java.rej
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/WorkerResource.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/WorkerResourceManager.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/rm/YarnTajoResourceManager.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorker.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoWorkerManagerService.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/Task.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TaskRunner.java
patching file tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TaskRunnerManager.java
patching file tajo-core/tajo-core-backend/src/main/proto/QueryMasterProtocol.proto
patching file tajo-core/tajo-core-backend/src/main/proto/TajoMasterProtocol.proto
patching file tajo-core/tajo-core-backend/src/main/proto/TajoWorkerProtocol.proto
patching file tajo-core/tajo-core-backend/src/main/resources/tajo-default.xml
patching file tajo-core/tajo-core-backend/src/main/resources/webapps/admin/cluster.jsp
Hunk #1 succeeded at 34 (offset 19 lines).
Hunk #2 succeeded at 84 (offset 19 lines).
patching file tajo-core/tajo-core-backend/src/main/resources/webapps/admin/index.jsp
Hunk #1 succeeded at 20 with fuzz 1 (offset 19 lines).
Hunk #2 succeeded at 134 (offset 19 lines).
Hunk #3 succeeded at 142 (offset 19 lines).
patching file tajo-core/tajo-core-backend/src/main/resources/webapps/admin/query.jsp
Hunk #1 succeeded at 20 with fuzz 1 (offset 19 lines).
Hunk #2 succeeded at 37 (offset 19 lines).
Hunk #3 succeeded at 73 (offset 19 lines).
Hunk #4 succeeded at 105 (offset 19 lines).
patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/env.jsp
Hunk #1 succeeded at 39 (offset 19 lines).
patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/index.jsp
Hunk #1 succeeded at 29 (offset 19 lines).
Hunk #2 succeeded at 55 (offset 19 lines).
Hunk #3 succeeded at 82 (offset 19 lines).
Hunk #4 succeeded at 97 (offset 19 lines).
Hunk #5 succeeded at 108 (offset 19 lines).
Hunk #6 succeeded at 134 (offset 19 lines).
patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/querydetail.jsp
Hunk #1 succeeded at 32 (offset 19 lines).
patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/queryplan.jsp
Hunk #1 succeeded at 39 (offset 19 lines).
patching file tajo-core/tajo-core-backend/src/main/resources/webapps/worker/querytasks.jsp
Hunk #1 succeeded at 46 (offset 19 lines).
patching file tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/TajoTestingCluster.java
patching file tajo-dist/src/main/bin/start-tajo.sh
patching file tajo-dist/src/main/bin/stop-tajo.sh
patching file tajo-dist/src/main/bin/tajo
patching file tajo-dist/src/main/bin/tajo-config.sh
patching file tajo-dist/src/main/conf/tajo-env.sh
{code}

> Separating QueryMaster and TaskRunner roles in worker
> -----------------------------------------------------
>
>                 Key: TAJO-275
>                 URL: https://issues.apache.org/jira/browse/TAJO-275
>             Project: Tajo
>          Issue Type: Improvement
>            Reporter: Keuntae Park
>            Assignee: Keuntae Park
>         Attachments: TAJO-275.patch
>
>
> In C++ implementation of Tajo worker,
> I think it is better to maintain Java code of QueryMaster and implement only TaskRunner
as C++ code, rather than implementing both QueryMaster and TaskRunner as C++ code.
> Accordingly, standby mode worker will have following 3 modes:
> 1) TaskRunner + QueryMaster (current implementation)
> 2) TaskRunner only (C++)
> 3) QueryMaster only (java)
> And,  most workers work as C++ TaskRunner, and 1 or 2 workers work as separate QueryMasters
(java process).
> For backward compatibility, default mode should be mode 1 (TaskRunner + QueryMaster)

> To achieve the goal, it needs to separate java TaskRunner and java QueryMaster Worker
first.
> (And then, implement C++ TaskRunner)



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message