drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jean-claude (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4858) REPEATED_COUNT on JSON containing an array of maps
Date Mon, 22 Aug 2016 15:06:20 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15430943#comment-15430943
] 

jean-claude commented on DRILL-4858:
------------------------------------

Seems like the org.apache.drill.exec.expr.EvaluationVisitor should handle the case of complex
repeated structures. At line 428 I've added an if statement to handle repeated complex
{code}
      } else if (!hasReadPath && complex && repeated) {
        JBlock eval = new JBlock();
        GetSetVectorHelper.read(e.getMajorType(),  vv1, eval, out, generator.getModel(), indexVariable);
        generator.getEvalBlock().add(eval);
{code}

This will produce the same code as when we are handling repeated int, varchar etc.

The only thing remaining is that the TypeHelper does not return a RepeatedMapHolder. So I
modified the template. I used to generate

{code}
  public static JType getHolderType(JCodeModel model, MinorType type, DataMode mode){
    switch (type) {
    case UNION:
      return model._ref(UnionHolder.class);
    case MAP:
    case LIST:
            return model._ref(ComplexHolder.class);
  {code}

Now the switch is like the other types int, varchar etc. That is it will return a RepeatedMapHolder.

{code}
  public static JType getHolderType(JCodeModel model, MinorType type, DataMode mode){
    switch (type) {
    case UNION:
      return model._ref(UnionHolder.class);
    case MAP:
        switch (mode) {
          case REQUIRED:
            return model._ref(ComplexHolder.class);
          case OPTIONAL:
            return model._ref(ComplexHolder.class);
          case REPEATED:
            return model._ref(RepeatedMapHolder.class);
        }
    case LIST:
        switch (mode) {
          case REQUIRED:
            return model._ref(ComplexHolder.class);
          case OPTIONAL:
            return model._ref(ComplexHolder.class);
          case REPEATED:
            return model._ref(RepeatedListHolder.class);
        }
{code}



> REPEATED_COUNT on JSON containing an array of maps
> --------------------------------------------------
>
>                 Key: DRILL-4858
>                 URL: https://issues.apache.org/jira/browse/DRILL-4858
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: jean-claude
>            Priority: Minor
>
> REPEATED_COUNT of JSON containing an array of map does not work.
> JSON file
> {code}
> drill$ cat /Users/jccote/repeated_count.json 
> {"intArray": [1,2,3,4], "mapArray": [{"name": "foo"},{"name": "foo"}], "label": "foo"}
> {"intArray": [1,2,3,4], "mapArray": [{"name": "foo"},{"name": "foo"}], "label": "foo"}
> {"intArray": [1,2,3,4], "mapArray": [{"name": "foo"},{"name": "foo"}], "label": "foo"}
> {"intArray": [1,2,3,4], "mapArray": [{"name": "foo"},{"name": "foo"}], "label": "foo"}
> {"intArray": [1,2,3,4], "mapArray": [{"name": "foo"},{"name": "foo"}], "label": "foo"}
> {"intArray": [1,2,3,4], "mapArray": [{"name": "foo"},{"name": "foo"}], "label": "foo"}
> {"intArray": [1,2,3,4], "mapArray": [{"name": "foo"},{"name": "foo"}], "label": "foo"}
> {"intArray": [1,2,3,4], "mapArray": [{"name": "foo"},{"name": "foo"}], "label": "foo"}
> {code}
> select
> {code}
> 0: jdbc:drill:zk=local> select repeated_count(mapArray) from dfs.`/Users/jccote/repeated_count.json`;
> {code}
> error
> {code}
> Error: SYSTEM ERROR: SchemaChangeException: Failure while trying to materialize incoming
schema.  Errors:
>  
> Error in expression at index -1.  Error: Missing function implementation: [repeated_count(MAP-REPEATED)].
 Full expression: --UNKNOWN EXPRESSION--..
> Fragment 0:0
> [Error Id: 1057bb8e-1cc4-4a9a-a748-3a6a14092858 on 192.168.1.3:31010] (state=,code=0)
> {code}
> Looking at the org.apache.drill.exec.expr.fn.impl.SimpleRepeatedFunctions
> Looks like it's not enabled yet. 
> {code}
>   // TODO - need to confirm that these work   SMP: They do not
>   @FunctionTemplate(name = "repeated_count", scope = FunctionTemplate.FunctionScope.SIMPLE)
>   public static class RepeatedLengthMap implements DrillSimpleFunc {
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message