drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben-Zvi <...@git.apache.org>
Subject [GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...
Date Tue, 26 Sep 2017 01:28:17 GMT
Github user Ben-Zvi commented on a diff in the pull request:

    https://github.com/apache/drill/pull/958#discussion_r140869257
  
    --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java
---
    @@ -22,16 +22,55 @@
     import javax.annotation.concurrent.ThreadSafe;
     
     import org.apache.drill.exec.exception.OutOfMemoryException;
    +import org.apache.drill.exec.util.AssertionUtil;
     
    +import com.google.common.annotations.VisibleForTesting;
     import com.google.common.base.Preconditions;
     
     /**
      * Provides a concurrent way to manage account for memory usage without locking. Used
as basis for Allocators. All
      * operations are threadsafe (except for close).
      */
     @ThreadSafe
    -class Accountant implements AutoCloseable {
    -  // private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(Accountant.class);
    +@VisibleForTesting
    +public class Accountant implements AutoCloseable {
    +  private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(Accountant.class);
    +
    +  // See DRILL-5808
    +  // Allow a "grace margin" above the allocator limit for those operators
    +  // that are trying to stay within the limit, but are hindered by the
    +  // current power-of-two allocations and random vector doublings. Instead
    +  // of failing the query, the accountant allows an excess allocation within
    +  // the grace, but issues a warning to the log.
    +  //
    +  // Grace is allowed only when requested, and then only in production
    +  // code, or if enabled in debug code by setting the following system
    +  // property.
    +
    +  public static final String STRICT_ALLOCATOR = "drill.memory.strict.allocator";
    --- End diff --
    
    Why is "strictness" a system property ?   Should users be able to turn it on during a
session (e.g., following an OOM) ?  


---

Mime
View raw message