tajo-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TAJO-1397) Resource allocation should be fine grained.
Date Mon, 20 Jul 2015 07:33:04 GMT

    [ https://issues.apache.org/jira/browse/TAJO-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633120#comment-14633120
] 

ASF GitHub Bot commented on TAJO-1397:
--------------------------------------

Github user hyunsik commented on the pull request:

    https://github.com/apache/tajo/pull/608#issuecomment-122786068
  
    LGTM. Here is my +1.


> Resource allocation should be fine grained.
> -------------------------------------------
>
>                 Key: TAJO-1397
>                 URL: https://issues.apache.org/jira/browse/TAJO-1397
>             Project: Tajo
>          Issue Type: Improvement
>          Components: QueryMaster, resource manager, Worker
>            Reporter: Hyunsik Choi
>            Assignee: Jinho Kim
>             Fix For: 0.11.0
>
>         Attachments: ResoruceSequence.jpg, TAJO-1397.patch, TAJO-1397_2.patch, TAJO-1397_3.patch,
TAJO-1397_4.patch, old_resource_circuit.png, resource_circuit.png
>
>
> See the comment:
> https://issues.apache.org/jira/browse/TAJO-540?focusedCommentId=14359478&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14359478
> From the discussion in TAJO-540
> {quote}
> In general, query (or job) scheduler aims at the maximum resource utilization. For multi-tenancy,
we also need to consider the fairness for multiple users (or queries). BTW, the maximum resource
utilization and fairness are usually conflict to each other in many cases. To mitigate this
problem, many scheduler seems to use preemption approach.
> In this point, our resource and scheduler system has the following problems:
>  * A query exclusively uses allocated resources at the first time until the query is
completed or failed.
>  * There is no mechanism to deallocate resources during query processing.
>  * Preempt is also not allowed.
> To achieve the multi tenancy, we should change our resource circulation. Especially,
resource allocation must be fine grained instead of per query.
> So, I'll create a jira issue to change the resource circulation. We have to do this issue
firstly in my opinion. If we achieve this, implementing multi-tenant scheduler would be much
easier than now. It would be a good starting point of this issue.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message