drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4503) Schema change exception even with all_text_mode enabled
Date Mon, 14 Mar 2016 02:59:33 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192692#comment-15192692
] 

ASF GitHub Bot commented on DRILL-4503:
---------------------------------------

Github user amansinha100 commented on a diff in the pull request:

    https://github.com/apache/drill/pull/428#discussion_r55952285
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
---
    @@ -402,7 +402,8 @@ private void writeDataAllText(MapWriter map, FieldSelection selection,
boolean m
             handleString(parser, map, fieldName);
             break;
           case VALUE_NULL:
    -        // do nothing as we don't have a type.
    --- End diff --
    
    Good point...these would fail to read due to the complex type field under all_text_mode.
 Let me see if there's a common solution to both types of use cases (the ones in my unit tests
and the ones here)..


> Schema change exception even with all_text_mode enabled
> -------------------------------------------------------
>
>                 Key: DRILL-4503
>                 URL: https://issues.apache.org/jira/browse/DRILL-4503
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - JSON
>    Affects Versions: 1.6.0
>            Reporter: Aman Sinha
>            Assignee: Hanifi Gunes
>             Fix For: 1.7.0
>
>         Attachments: mostlynulls_1.json
>
>
> Both HashAggregate and StreamingAggregate encounter schema change error whey querying
a JSON file with non-null values for column 'a' and many null values for column 'c'.
> This occurs even when all_text_mode is enabled, which seems counterintuitive since once
all_text_mode is enabled, everything (including nulls) should be treated as varchar and one
would expect no schema change errors.  
> Here are some example queries that encounter this error: 
> {noformat}
> 0: jdbc:drill:zk=local> select a, c from dfs.`mostlynulls_1.json` group by a, c;
> Error: UNSUPPORTED_OPERATION ERROR: Hash aggregate does not support schema changes
> 0: jdbc:drill:zk=local> alter session set `store.json.all_text_mode` = true;
> +-------+------------------------------------+
> |  ok   |              summary               |
> +-------+------------------------------------+
> | true  | store.json.all_text_mode updated.  |
> +-------+------------------------------------+
> 1 row selected (0.15 seconds)
> 0: jdbc:drill:zk=local> select a, c from dfs.`mostlynulls_1.json` group by a, c;
> Error: UNSUPPORTED_OPERATION ERROR: Hash aggregate does not support schema changes
> 0: jdbc:drill:zk=local> select min(a), min(c) from dfs.`mostlynulls_1.json`;
> Error: UNSUPPORTED_OPERATION ERROR: Streaming aggregate does not support schema changes
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message