phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maryann Xue (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-953) Support UNNEST for ARRAY
Date Wed, 19 Aug 2015 18:51:45 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14703555#comment-14703555
] 

Maryann Xue commented on PHOENIX-953:
-------------------------------------

Thanks for updating the patch, [~Dumindux]! Actually your original patch (v1) was already
good enough for the runtime part, except that you should return a new tuple for each array
element. So the first two steps in patch v1 were perfectly fine: 1) get the tuple from the
"delegate"; 2) retrieve the array value using standard expression.evaluate() interface. I
don't see why we should copy that ProjectedColumnExpression logic into the iterator, and since
the array expression is included in the UnnestExpression this step can be removed.
So the iterator should be something like:
{code}
+
+    public class UnnestArrayResultIterator implements ResultIterator {
+        private TupleProjector projector;
+        private Tuple current;
+
+        public UnnestArrayResultIterator(ResultIterator iterator) {
+            this.delegate = iterator;
+            this.projector = new TupleProjector(new Expression[] {unnestExpression}); //
TupleProjector should provide a new constructor that deducts the KeyValueSchema based on the
input expressions.
+        }
+
+        @Override
+        public Tuple next() throws SQLException {
+            if (closed)
+                return;
+            while (unnestExpression.isEnd()) {
+                current = delegate.next();
+                if (current == null) {
+                    this.closed = true;
+                    return null;
+                }
+                // this is to check empty arrays.
+                if (unnestExpression.evaluate(current)) {
+                    unnestExpression.rewind();
+                }
+            }
+            return projector.projectResults(current);
+        }
+
+        // other functions
+    }
{code}

And the UnnestExpression should be like:
{code}
+
+public class UnnestExpression extends BaseCompoundExpression {
+    private int index = -1;
+    private int length = -1;
+
+    // constructors
+
+    public isEnd() {
+        return index >= length;
+    }
+
+    public rewind() {
+        index = 0;
+    }
+
+    @Override
+    public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) {
+        if (isEnd()) {
+            if (!children.get(0).evaluate(tuple, ptr))
+                return false;
+            length = PArrayDataType.getArrayLength(ptr, getDataType(), getMaxLength());
+            index = 0;
+        }
+
+        if (isEnd())
+            return false;
+
+        PArrayDataType.positionAtArrayElement(ptr, index++, getDataType(), getMaxLength());
+        return true;
+    }
+
+    // other functions
{code}

The compiler support for UNNEST will be done by calcite in the Phoenix/Calcite model, so you
can remove all other changes from the compiler (except the parser, am I right, [~jamestaylor]?).
As to the test cases, we need two types of tests:
1. The end-to-end query tests, which will be "ignored" for now.
2. The UnnestArrayQueryPlan unit tests to test the behavior of query plan and result iterator
only. 

> Support UNNEST for ARRAY
> ------------------------
>
>                 Key: PHOENIX-953
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-953
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: James Taylor
>            Assignee: Dumindu Buddhika
>         Attachments: PHOENIX-953-v1.patch, PHOENIX-953-v2.patch
>
>
> The UNNEST built-in function converts an array into a set of rows. This is more than
a built-in function, so should be considered an advanced project.
> For an example, see the following Postgres documentation: http://www.postgresql.org/docs/8.4/static/functions-array.html
> http://www.anicehumble.com/2011/07/postgresql-unnest-function-do-many.html
> http://tech.valgog.com/2010/05/merging-and-manipulating-arrays-in.html
> So the UNNEST is a way of converting an array to a flattened "table" which can then be
filtered on, ordered, grouped, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message