pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park" <piaozhe...@gmail.com>
Subject Re: Review Request 20761: PIG-3860 Refactor PigStatusReporter and PigLogger for non-MR execution engine
Date Wed, 30 Apr 2014 16:43:24 GMT


> On April 30, 2014, 2:30 p.m., Rohini Palaniswamy wrote:
> > src/org/apache/pig/tools/pigstats/PigStatusReporter.java, line 54
> > <https://reviews.apache.org/r/20761/diff/2/?file=569672#file569672line54>
> >
> >     Can you make a amend checkin renaming this to destroy?

Sure, I will. Thanks for catching that.


> On April 30, 2014, 2:30 p.m., Rohini Palaniswamy wrote:
> > shims/src/hadoop23/org/apache/pig/backend/hadoop/executionengine/shims/TaskContext.java,
line 104
> > <https://reviews.apache.org/r/20761/diff/2/?file=569655#file569655line104>
> >
> >     We could have avoided the if/else and the shims class as well by having this
class as abstract and different context types as implementations. But I guess this is good
for now and we can refactor it later in a separate jira.

Good suggestion. I'll follow up with a new jira.


- Cheolsoo


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20761/#review41831
-----------------------------------------------------------


On April 28, 2014, 9:32 p.m., Cheolsoo Park wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20761/
> -----------------------------------------------------------
> 
> (Updated April 28, 2014, 9:32 p.m.)
> 
> 
> Review request for pig, Daniel Dai and Rohini Palaniswamy.
> 
> 
> Bugs: PIG-3860
>     https://issues.apache.org/jira/browse/PIG-3860
> 
> 
> Repository: pig-git
> 
> 
> Description
> -------
> 
> The patch includes the following changes-
> 
> 1) Introduce a generic class called TaskContext<?>.
>    - For MR, it holds TaskInputOutputContext
>    - For Tez, it holds TezProcessorContext
>    - For Fetch, it holds FetchContext
>    Note that it is under the shims package because TezProcessorContext is not visible
when -Dhadoopversion=20.
> 2) Refactor PigStatusReporter and PigHadoopLogger by using generic TaskContext instead
of MR-specific TaskInputOutputContext.
> 3) Whitespace fixes.
> 
> The diff file in this RB request is for tez branch. I'll post another to the jira for
trunk.
> 
> 
> Diffs
> -----
> 
>   shims/src/hadoop20/org/apache/pig/backend/hadoop/executionengine/shims/TaskContext.java
e69de29 
>   shims/src/hadoop23/org/apache/pig/backend/hadoop/executionengine/shims/TaskContext.java
e69de29 
>   src/org/apache/pig/TypedOutputEvalFunc.java 5c73bf5 
>   src/org/apache/pig/backend/hadoop/executionengine/fetch/FetchLauncher.java 689e424

>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MapReducePOStoreImpl.java
215ff89 
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigCombiner.java b41648f

>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigGenericMapBase.java
c52603f 
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigGenericMapReduce.java
bb10f0c 
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigHadoopLogger.java
4db4a11 
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigMapReduceCounter.java
2695be3 
>   src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigRecordReader.java
0c221e0 
>   src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/PigLogger.java 4ef44e8

>   src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/POUserFunc.java
16b6fb9 
>   src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/util/MonitoredUDFExecutor.java
8fc0c65 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java 7c8286a 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/PigTezLogger.java c835e93 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezStatusReporter.java e1b6157

>   src/org/apache/pig/tools/counters/PigCounterHelper.java 0023035 
>   src/org/apache/pig/tools/pigstats/PigStatusReporter.java 5b1bc42 
> 
> Diff: https://reviews.apache.org/r/20761/diff/
> 
> 
> Testing
> -------
> 
> All unit tests pass.
> All e2e tests pass except Warning_4 (known - PIG-3739).
> 
> 
> Thanks,
> 
> Cheolsoo Park
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message