tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jihoon Son (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (TAJO-385) Refactoring TaskScheduler to assign multiple fragments
Date Sun, 15 Dec 2013 07:52:06 GMT

     [ https://issues.apache.org/jira/browse/TAJO-385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jihoon Son updated TAJO-385:
----------------------------

    Attachment: TAJO-385_3.patch

I attached the third patch after rebasing.
In this patch, I also modified the task size configuration as follows.
{noformat}
<property>
  <name>tajo.task.size-mb.default</name>
  <value>task_size_in_mb</value>
</property>
{noformat}

> Refactoring TaskScheduler to assign multiple fragments
> ------------------------------------------------------
>
>                 Key: TAJO-385
>                 URL: https://issues.apache.org/jira/browse/TAJO-385
>             Project: Tajo
>          Issue Type: Improvement
>          Components: query master
>    Affects Versions: 0.8-incubating
>            Reporter: Jihoon Son
>            Assignee: Jihoon Son
>         Attachments: TAJO-385.patch, TAJO-385_2.patch, TAJO-385_3.patch
>
>
> In the current implementation, each task processes only one fragment.
> However, processing multiple fragments in a task will increase the query processing performance
according to the storage layout and the user queries.
> In this issue, TaskScheduler is refactored to enable assigning multiple fragments to
each task.
> Followings should be contained.
> * Schedule Fragments instead of QueryUnits in TaskScheduler
> ** The QueryUnit creation is postponed until TaskScheduler receives task requests from
workers.
> ** When TaskScheduler receives task requests from workers, it dynamically creates an
QueryUnit and assigns one or more fragments.
> ** The fragment scheduling should take into account the disk load balancing.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message