pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Wagner" <wagner.mar...@gmail.com>
Subject Re: Review Request 13950: Tez backend layout
Date Tue, 03 Sep 2013 21:48:19 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13950/#review25860
-----------------------------------------------------------


This looks like a good layout to start with. There are a couple things that seem to be carried
over from the 'pigeon' work, that I'm not sure are applicable.


src/org/apache/pig/backend/hadoop/executionengine/tez/MRROptimizer.java
<https://reviews.apache.org/r/13950/#comment50425>

    I think this is related to the 'pigeon' flow of compilation. Is it still applicable for
the new approach?



src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java
<https://reviews.apache.org/r/13950/#comment50426>

    Do we still need this when we have the DAG api from Tez? It seems strange to wrap Tez
things in legacy MR APIs. If this is really needed, is it general enough to be included in
the Tez project?



src/org/apache/pig/backend/hadoop/executionengine/tez/TezJobControlCompiler.java
<https://reviews.apache.org/r/13950/#comment50427>

    Same with above. Do we still need this? Perhaps these should be left out until the details
of TezLauncher are worked out. We can add them later if it turns out we need them


- Mark Wagner


On Sept. 3, 2013, 9:14 p.m., Cheolsoo Park wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13950/
> -----------------------------------------------------------
> 
> (Updated Sept. 3, 2013, 9:14 p.m.)
> 
> 
> Review request for pig.
> 
> 
> Bugs: PIG-3448
>     https://issues.apache.org/jira/browse/PIG-3448
> 
> 
> Repository: pig-git
> 
> 
> Description
> -------
> 
> Adds skeleton classes that I think we need to implement for Tez backend.
> 
> 
> Diffs
> -----
> 
>   build.xml 7e22192 
>   ivy.xml aa8f90a 
>   ivy/libraries.properties 474edbd 
>   src/META-INF/services/org.apache.pig.ExecType 7065767 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/DagUtils.java PRE-CREATION 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/MRROptimizer.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/MapOper.java PRE-CREATION 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/ReduceOper.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezExecType.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezExecutionEngine.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java PRE-CREATION 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezJobControlCompiler.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezOpPlanVisitor.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezOperPlan.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezOperator.java PRE-CREATION

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezPrinter.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/13950/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Cheolsoo Park
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message