pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy" <rohini.adi...@gmail.com>
Subject Re: Review Request 21465: PIG-3918 Implement PPNL for Tez mode (Pig side changes)
Date Thu, 15 May 2014 22:17:26 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21465/#review43167
-----------------------------------------------------------



src/org/apache/pig/backend/hadoop/executionengine/tez/POSimpleTezLoad.java
<https://reviews.apache.org/r/21465/#comment77231>

    Can we remove this constructor?



src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java
<https://reviews.apache.org/r/21465/#comment77232>

    getDAGProgress



src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java
<https://reviews.apache.org/r/21465/#comment77233>

    Shouldn't it be job.getMapredJobId()?



src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java
<https://reviews.apache.org/r/21465/#comment77230>

    Can we set id to that of the application (controlledjob.getMapredJobID()) as tts.getName()
is already vertex name?


- Rohini Palaniswamy


On May 14, 2014, 11:49 p.m., Cheolsoo Park wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21465/
> -----------------------------------------------------------
> 
> (Updated May 14, 2014, 11:49 p.m.)
> 
> 
> Review request for pig, Daniel Dai and Rohini Palaniswamy.
> 
> 
> Bugs: PIG-3918
>     https://issues.apache.org/jira/browse/PIG-3918
> 
> 
> Repository: pig-git
> 
> 
> Description
> -------
> 
> This patch makes PPNL work with Pig-on-Tez:
> 
> * Emit event notifications to PPNL in TezLauncher.
> * Add a TezJob handle to TezStats. PPNL needs to access to DAG status to report progress.
> * Along the way, two bug fixes-
>   * TezDagBuilder wasn't setting POLoad#lFile while constructing POSimpleTezLoad. This
causes NPE in PPNL.
>   * BaseOperatorPlan#connect() shouldn't add nulls to edge maps. This causes NPE in PPNL.
> * Fix unit tests. Some tests were failing after my changes because of ClassCastException
from TezScriptState to MRScriptState.
> 
> 
> Diffs
> -----
> 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/POSimpleTezLoad.java f5ea1a5

>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java 692cf08 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezJob.java da587f8 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezJobControl.java de01a9f 
>   src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java f5a8461 
>   src/org/apache/pig/newplan/BaseOperatorPlan.java d86b60b 
>   src/org/apache/pig/tools/pigstats/tez/TezStats.java f94b954 
>   test/org/apache/pig/test/TestAccumulator.java 16f067a 
>   test/org/apache/pig/test/TestBuiltin.java d33ed89 
>   test/org/apache/pig/test/TestLocalRearrange.java 7ffc872 
>   test/org/apache/pig/test/TestSkewedJoin.java 1ce3350 
> 
> Diff: https://reviews.apache.org/r/21465/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Cheolsoo Park
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message