asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Murtadha Hubail (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: [NO ISSUE][RT] Remove Hard Memory Limit in ByteArrayAccessib...
Date Sun, 15 Oct 2017 02:17:37 GMT
Murtadha Hubail has submitted this change and it was merged.

Change subject: [NO ISSUE][RT] Remove Hard Memory Limit in ByteArrayAccessibleOutputStream
......................................................................


[NO ISSUE][RT] Remove Hard Memory Limit in ByteArrayAccessibleOutputStream

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Remove the hardcoded 64MB memory limit from
  ByteArrayAccessibleOutputStream as the limit
  shouldn't be controlled by this structure but
  rather by operators using it.

Change-Id: Ia88861c44802e64dbfceb5e8efc75d28bd54b501
Reviewed-on: https://asterix-gerrit.ics.uci.edu/2073
Tested-by: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Contrib: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Reviewed-by: Till Westmann <tillw@apache.org>
---
M hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java
1 file changed, 12 insertions(+), 11 deletions(-)

Approvals:
  Anon. E. Moose #1000171: 
  Till Westmann: Looks good to me, approved
  Jenkins: Verified; ; Verified

Objections:
  Jenkins: Violations found



diff --git a/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java
b/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java
index bf0e1dd..1a806cd 100644
--- a/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java
+++ b/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/util/ByteArrayAccessibleOutputStream.java
@@ -23,8 +23,7 @@
 
 public class ByteArrayAccessibleOutputStream extends ByteArrayOutputStream {
 
-    private static final int MAX_SIZE = 1024 * 1024 * 32;
-    private static final double BUFFER_INCREMENT_FACTOR = 1.5;
+    private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE;
 
     public ByteArrayAccessibleOutputStream() {
         super();
@@ -97,22 +96,24 @@
     private void grow(int minCapacity) {
         // overflow-conscious code
         int oldCapacity = buf.length;
-        if (oldCapacity == MAX_SIZE) {
-            throw new IllegalArgumentException("Buffer is too large...");
-        }
-        int newCapacity = Math.min((int) (oldCapacity * BUFFER_INCREMENT_FACTOR), MAX_SIZE);
+        // increase by a factor of 1.5
+        int newCapacity = oldCapacity + (oldCapacity >> 1);
         if (newCapacity - minCapacity < 0) {
             newCapacity = minCapacity;
         }
-        if (newCapacity < 0) {
-            if (minCapacity < 0) {
-                throw new OutOfMemoryError();
-            }
-            newCapacity = Integer.MAX_VALUE;
+        if (newCapacity - MAX_ARRAY_SIZE > 0) {
+            newCapacity = hugeCapacity(minCapacity);
         }
         buf = Arrays.copyOf(buf, newCapacity);
     }
 
+    private static int hugeCapacity(int minCapacity) {
+        if (minCapacity < 0) { // overflow
+            throw new RuntimeException("Memory allocation limit (" + MAX_ARRAY_SIZE + " bytes)
exceeded");
+        }
+        return (minCapacity > MAX_ARRAY_SIZE) ? Integer.MAX_VALUE : MAX_ARRAY_SIZE;
+    }
+
     /**
      * Return the current length of this stream (not capacity).
      */

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2073
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ia88861c44802e64dbfceb5e8efc75d28bd54b501
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <mhubail@apache.org>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>
Gerrit-Reviewer: Murtadha Hubail <mhubail@apache.org>
Gerrit-Reviewer: Till Westmann <tillw@apache.org>

Mime
View raw message