drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From arina-ielchiieva <...@git.apache.org>
Subject [GitHub] drill pull request #958: DRILL-5808: Reduce memory allocator strictness for ...
Date Wed, 11 Oct 2017 11:58:18 GMT
Github user arina-ielchiieva commented on a diff in the pull request:

    https://github.com/apache/drill/pull/958#discussion_r143976358
  
    --- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/Accountant.java
---
    @@ -22,16 +22,67 @@
     import javax.annotation.concurrent.ThreadSafe;
     
     import org.apache.drill.exec.exception.OutOfMemoryException;
    +import org.apache.drill.exec.util.AssertionUtil;
     
    +import com.google.common.annotations.VisibleForTesting;
     import com.google.common.base.Preconditions;
     
     /**
      * Provides a concurrent way to manage account for memory usage without locking. Used
as basis for Allocators. All
      * operations are threadsafe (except for close).
      */
     @ThreadSafe
    -class Accountant implements AutoCloseable {
    -  // private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(Accountant.class);
    +@VisibleForTesting
    +public class Accountant implements AutoCloseable {
    +  private static final org.slf4j.Logger logger = org.slf4j.LoggerFactory.getLogger(Accountant.class);
    +
    +  // See DRILL-5808
    +  // Allow a "grace margin" above the allocator limit for those operators
    +  // that are trying to stay within the limit, but are hindered by the
    +  // current power-of-two allocations and random vector doublings. Instead
    +  // of failing the query, the accountant allows an excess allocation within
    +  // the grace, but issues a warning to the log.
    +  //
    +  // Normally, the "grace margin" (lenient) allocation is allowed only for
    +  // operators that request leniency (because they attempt to manage memory)
    +  // and only in production mode. Leniency is usually denied in a debug
    +  // run (assertions enabled.) Use the following system option to force
    +  // allowing lenient allocation in a debug run (or, to disable lenient
    +  // allocation in a production run.)
    +  //
    +  // Force allowing lenient allocation:
    +  //
    +  // -Ddrill.memory.lenient.allocator=false
    +  //
    +  // Force strict allocation;
    +  //
    +  // -Ddrill.memory.lenient.allocator=false
    +
    +  public static final String ALLOW_LENIENT_ALLOCATION = "drill.memory.lenient.allocator";
    --- End diff --
    
    `drill.memory.lenient.allocation`?


---

Mime
View raw message