tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "JaeHwa Jung (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TAJO-184) Refactor GlobalPlanner and global plan data structure
Date Mon, 16 Sep 2013 08:29:52 GMT

    [ https://issues.apache.org/jira/browse/TAJO-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768155#comment-13768155
] 

JaeHwa Jung commented on TAJO-184:
----------------------------------

'mvn clean install' fails with the following error:

{code:xml}
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile
(default-compile) on project tajo-common: Compilation failure: Compilation failure:
[ERROR] /tajo-common/src/main/java/org/apache/tajo/ExecutionBlockId.java:[21,16] cannot find
symbol
[ERROR] symbol  : class Objects
[ERROR] location: package java.util
[ERROR] /tajo-common/src/main/java/org/apache/tajo/ExecutionBlockId.java:[68,11] cannot find
symbol
[ERROR] symbol  : variable Objects
[ERROR] location: class org.apache.tajo.ExecutionBlockId
{code}

For reference, I used the lastest source.
                
> Refactor GlobalPlanner and global plan data structure
> -----------------------------------------------------
>
>                 Key: TAJO-184
>                 URL: https://issues.apache.org/jira/browse/TAJO-184
>             Project: Tajo
>          Issue Type: Improvement
>          Components: master, physical operator, planner/optimizer
>            Reporter: Hyunsik Choi
>            Assignee: Hyunsik Choi
>             Fix For: 0.2-incubating
>
>         Attachments: TAJO-184.patch
>
>
> Above all, I'm sorry for submitting a big patch. This patch modifies and refactors broadly
global planning, logical planning, and physical planning parts. It was hard to separate this
issue into smaller issues.
> Especially, this patch primarily rewrites GlobalPlanner and MasterPlan (global plan data
structure) as follows:
>  * Removed GlobalPlanOptimizer
>  * Added DirectedGraph interface, SimpleDirectedGraph concret class, and a visitor class
to visit a graph in post-order traverse way.
>  * Improved MasterPlan by using new graph API
>   ** query block graphs and an execution block graph are represented by SimpleDirectedGraph.
>   ** Now, we can traverse above graphs easily by using graph APIs.
>   ** Added DataChannel class to represent a data flow between execution blocks.
>  * MasterPlan.toString() prints a text graph to represent relationships among execution
blocks and a distributed plan.
>  * Add more sophisticated explain feature for a distributed plan and logical plan. It
is very useful for plan debugging.
>  * Now, the limit operator is pushed down to child execution block.
>   ** So, the intermediate data volume of a sort query with limit is reduced significantly.
>  * TableSubQuery (inline view) is supported. It follows SQL standards. So, you can do
a query as follows:
> {code}
> SELECT *
> FROM
> (
>     SELECT
>         l_orderkey,
>         l_partkey,
>         url
>     FROM
>         (
>           SELECT
>             l_orderkey,
>             l_partkey,
>             CASE
>               WHEN
>                 l_partkey IS NOT NULL THEN ''
>               WHEN l_orderkey = 1 THEN '1'
>             ELSE
>               '2'
>             END AS url
>           FROM
>             lineitem
>         ) res1
>         JOIN
>         (
>           SELECT
>             *
>           FROM
>             part
>         ) res2
>         ON l_partkey = p_partkey
> ) result
> {code}
> In addition, I've refactored as follows:
>  * Column has a qualifier name.
>  * Improved Schema to deal with qualified column names
>  * When a TableDesc instance is retrieved, it is forced to have qualifier columns.
>  * Fixed TAJO-162 bug.
>  * Lots of trivial improvement and refactors.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message