Mailing-List: contact dev-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
From: paul-rogers <git@git.apache.org>
To: dev@drill.apache.org
Reply-To: dev@drill.apache.org
References: <git-pr-1075-drill@git.apache.org>
In-Reply-To: <git-pr-1075-drill@git.apache.org>
Subject: [GitHub] drill pull request #1075: DRILL-6030: Managed sort should minimize number of...
Content-Type: text/plain
Message-Id: <20171219180424.2CD75DFFCF@git1-us-west.apache.org>
Date: Tue, 19 Dec 2017 18:04:24 +0000 (UTC)
archived-at: Tue, 19 Dec 2017 18:04:26 -0000

Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1075#discussion_r157830252
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/managed/SortConfig.java ---
    @@ -84,7 +85,7 @@ public SortConfig(DrillConfig config) {
         if (limit > 0) {
           mergeLimit = Math.max(limit, MIN_MERGE_LIMIT);
         } else {
    -      mergeLimit = Integer.MAX_VALUE;
    +      mergeLimit = DEFAULT_MERGE_LIMIT;
    --- End diff --
    
    The merge limit is already a config option. (I'd forgotten about that.) The comment on the config option says "Limit on the number of spilled batches that can be merged in a single pass." So, let's just set that default (in `drill-override-conf`) to your new value of 128 and leave the code here unchanged.


---