pig-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From knogu...@apache.org
Subject svn commit: r1862506 - in /pig/trunk: CHANGES.txt src/org/apache/pig/impl/util/SpillableMemoryManager.java
Date Wed, 03 Jul 2019 17:53:21 GMT
Author: knoguchi
Date: Wed Jul  3 17:53:20 2019
New Revision: 1862506

URL: http://svn.apache.org/viewvc?rev=1862506&view=rev
Log:
PIG-5385: Skip calling extra gc() before spilling large bag when unnecessary (knoguchi)

Modified:
    pig/trunk/CHANGES.txt
    pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java

Modified: pig/trunk/CHANGES.txt
URL: http://svn.apache.org/viewvc/pig/trunk/CHANGES.txt?rev=1862506&r1=1862505&r2=1862506&view=diff
==============================================================================
--- pig/trunk/CHANGES.txt (original)
+++ pig/trunk/CHANGES.txt Wed Jul  3 17:53:20 2019
@@ -26,6 +26,8 @@ PIG-5282: Upgade to Java 8 (satishsaley
  
 IMPROVEMENTS
 
+PIG-5385: Skip calling extra gc() before spilling large bag when unnecessary (knoguchi)
+
 PIG-5389: Passing null to REPLACE udf preventing JIT optimization (knoguchi)
 
 PIG-5382: Log the name when POStore fails on write (knoguchi)

Modified: pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java
URL: http://svn.apache.org/viewvc/pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java?rev=1862506&r1=1862505&r2=1862506&view=diff
==============================================================================
--- pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java (original)
+++ pig/trunk/src/org/apache/pig/impl/util/SpillableMemoryManager.java Wed Jul  3 17:53:20
2019
@@ -293,7 +293,8 @@ public class SpillableMemoryManager impl
                     // Do not invoke extraGC for GroupingSpillable. Its size will always
exceed
                     // extraGCSpillSizeThreshold and the data is always strong referenced.
                     if( !extraGCCalled && extraGCSpillSizeThreshold != 0
-                        && toBeFreed > extraGCSpillSizeThreshold  && !isGroupingSpillable)
{
+                        && toBeFreed > extraGCSpillSizeThreshold  && !isGroupingSpillable
+                        && n.getType().equals(MemoryNotificationInfo.MEMORY_THRESHOLD_EXCEEDED))
{
                         log.debug("Single spillable has size " + toBeFreed + "bytes. Calling
extra gc()");
                         // this extra assignment to null is needed so that gc can free the
                         // spillable if nothing else is pointing at it



Mime
View raw message