pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai" <dai...@gmail.com>
Subject Re: Review Request 17876: [PIG-3456] Reduce threadlocal conf access in backend for each record
Date Thu, 23 Oct 2014 21:08:17 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17876/#review58103
-----------------------------------------------------------


Thanks Rohini, the same fix should also goes to CombinerPackager.java, POCollectedGroup.java,
PODistinct.java, POPackage.java, POPartialAgg.java, POSort.java, Packager.java.

- Daniel Dai


On Oct. 23, 2014, 9:02 p.m., Rohini Palaniswamy wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17876/
> -----------------------------------------------------------
> 
> (Updated Oct. 23, 2014, 9:02 p.m.)
> 
> 
> Review request for pig.
> 
> 
> Bugs: PIG-3456
>     https://issues.apache.org/jira/browse/PIG-3456
> 
> 
> Repository: pig
> 
> 
> Description
> -------
> 
> 1) DefaultTuple has a protected boolean isNull = false; which is never used. Removing
this gives ~3-5% improvement for big jobs
> 2) Config checking with ThreadLocal conf is repeatedly done for each record. For eg:
createDataBag in POCombinerPackage. But initialized only for first time in other places like
POPackage, POJoinPackage, etc.
> 
> 
> Diffs
> -----
> 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/PigConfiguration.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/CombinerPackager.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POCollectedGroup.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/PODistinct.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPackage.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POSort.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/Packager.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/Distinct.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/DefaultAbstractBag.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/DefaultTuple.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/InternalDistinctBag.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/SelfSpillBag.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestPOPartialAgg.java
1633945 
>   http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestTuple.java 1633945

> 
> Diff: https://reviews.apache.org/r/17876/diff/
> 
> 
> Testing
> -------
> 
> Full suite of unit tests pass
> 
> 
> Thanks,
> 
> Rohini Palaniswamy
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message