drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Dunning (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3462) There appears to be no way to have complex intermediate state
Date Mon, 06 Jul 2015 20:55:05 GMT
Ted Dunning created DRILL-3462:
----------------------------------

             Summary: There appears to be no way to have complex intermediate state
                 Key: DRILL-3462
                 URL: https://issues.apache.org/jira/browse/DRILL-3462
             Project: Apache Drill
          Issue Type: Bug
            Reporter: Ted Dunning


After spending several frustrating days on the problem (see also DRILL-3461), it appears that
there is no viable idiom for building an aggregator that has internal state that is anything
more than a scalar.

What is needed is:

1) The ability to allocate a Repeated* type for use in a Workspace variables.  Currently,
new works to get the basic structure, but there is no good way to allocate the corresponding
vector.

2) The ability to use and to allocate a ComplexWriter in the Workspace variables.

3) The ability to write a UDAF that supports multi-phase aggregation.  It would be just fine
if I simply have to write a combine method on my UDAF class.  I don't think that there is
any way to infer such a combiner from the parameters and workspace variables.  An alternative
API would be to have a form of the output function that is given an Iterable<OutputClass>,
but that is probably much less efficient than simply having a combine method that is called
repeatedly.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message