tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jihoon Son (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TAJO-385) Refactoring TaskScheduler to assign multiple fragments
Date Thu, 05 Dec 2013 12:11:35 GMT
Jihoon Son created TAJO-385:
-------------------------------

             Summary: Refactoring TaskScheduler to assign multiple fragments
                 Key: TAJO-385
                 URL: https://issues.apache.org/jira/browse/TAJO-385
             Project: Tajo
          Issue Type: Improvement
          Components: master
    Affects Versions: 0.8-incubating
            Reporter: Jihoon Son
            Assignee: Jihoon Son


In the current implementation, each task processes only one fragment.
However, processing multiple fragments in a task will increase the query processing performance
according to the storage layout and the user queries.
In this issue, TaskScheduler is refactored to enable assigning multiple fragments to each
task.
Followings should be contained.
* Schedule Fragments instead of QueryUnits in TaskScheduler
** The QueryUnit creation is postponed until TaskScheduler receives task requests from workers.
** When TaskScheduler receives task requests from workers, it dynamically creates an QueryUnit
and assigns one or more fragments.
** The fragment scheduling should take into account the disk load balancing.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message