phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [phoenix] dbwong commented on a change in pull request #463: Phoenix stats Initial Commit
Date Fri, 05 Apr 2019 21:43:40 GMT
dbwong commented on a change in pull request #463: Phoenix stats Initial Commit
URL: https://github.com/apache/phoenix/pull/463#discussion_r272752994
 
 

 ##########
 File path: phoenix-core/src/main/java/org/apache/phoenix/compile/ScanRanges.java
 ##########
 @@ -694,4 +694,136 @@ public TimeRange getRowTimestampRange() {
         return rowTimestampRange;
     }
 
+    /**
+     * Produces the list of KeyRanges representing the fully qualified row key by calling
into ScanUtil.setKey
+     * repeatedly for every combination of KeyRanges in the ranges field. The bounds will
be set according to the
+     * properties of the setKey method.
+     * 
+     * @return list of KeyRanges representing the fully qualified rowkey, coalesced
+     */
+    public List<KeyRange> getRowKeyRanges() {
+        // If its scanRanges are everything or nothing, then we short circuit and leave
+        // as schema is not filled in
+        if (isEverything()) { return Lists.newArrayList(KeyRange.EVERYTHING_RANGE); }
+        if (isDegenerate()) { return Lists.newArrayList(KeyRange.EMPTY_RANGE); }
+
+        List<KeyRange> queryRowKeyRanges = Lists.newArrayListWithExpectedSize(this.ranges.size());
+
+        // Point lookups are stored in the first range as a whole and already a rowkey
+        // see ScanRanges.getPointLookupKeyIterator
+        if (this.isPointLookup()) {
+            queryRowKeyRanges.addAll(this.getRanges().get(0));
+        } else {
+            // If scanRanges.ranges has no information then should be in the scanRanges.scanRange
+            if (this.getRanges().size() == 0) {
+                queryRowKeyRanges.add(this.getScanRange());
+            } else { // We have a composite key need the row key from the combination
+                // make a copy of ranges as we may add items to fully qualify our rowkey
+                List<List<KeyRange>> newRanges = new ArrayList<>(this.getRanges());
+
+                int[] slotSpans = this.getSlotSpans();
+
+                // If the ranges here do not qualify all the keys then those keys are unbound
+                if (newRanges.size() < schema.getMaxFields()) {
+                    int originalSize = newRanges.size();
+                    for (int i = 0; i < schema.getMaxFields() - originalSize; i++) {
+                        newRanges.add(Lists.newArrayList(KeyRange.EVERYTHING_RANGE));
+                    }
+                    slotSpans = new int[schema.getMaxFields()];
+                    System.arraycopy(this.getSlotSpans(), 0, slotSpans, 0, this.getSlotSpans().length);
+                }
+
+                // Product to bound our counting loop for safety
+                int rangesPermutationsCount = 1;
+                for (int i = 0; i < newRanges.size(); i++) {
+                    rangesPermutationsCount *= newRanges.get(i).size();
+                }
+
+                // Have to construct the intersection
+                List<KeyRange> expandedRanges = Lists.newArrayListWithCapacity(rangesPermutationsCount);
+                int[] positions = new int[slotSpans.length];
+
+                int maxKeyLength = SchemaUtil.getMaxKeyLength(schema, newRanges);
+                byte[] keyBuffer = new byte[maxKeyLength];
+
+                // Counting Loop
+                int count = 0;
+                while (count < rangesPermutationsCount) {
 
 Review comment:
   I think using in memory is fine for a first approach.  I discussed this briefly with @BinShi-SecularBird
and we felt that even large number of ranges should be okay for a first pass.  Some estimates
with a 100 byte key, and 100 keys, with 100 settings, is ~1 mb. Is this a current concern
for the memory footprint?  Second with an iterator approach the next steps of intersecting
with region boundaries and with guidepost tree structure may be less efficient so we are trading
memory for execution time.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message