drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3562) Query fails when using flatten on JSON data where some documents have an empty array
Date Mon, 09 Jan 2017 03:59:58 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15810598#comment-15810598
] 

ASF GitHub Bot commented on DRILL-3562:
---------------------------------------

Github user amansinha100 commented on a diff in the pull request:

    https://github.com/apache/drill/pull/713#discussion_r95100719
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
---
    @@ -59,6 +59,12 @@
       private final boolean readNumbersAsDouble;
     
       /**
    +   * Collection for tracking empty array writers during reading
    +   * and storing them for initializing empty arrays
    +   */
    +  private final Set<ListWriter> emptyArrayWritersSet = Sets.newHashSet();
    --- End diff --
    
    Any reason why this needs to be HashSet rather than just a List ?  The HashSet may change
the insertion order, so if you had 2 or more empty arrays in the same Json doc within one
list,  the output of Flatten could end up changing the order.  


> Query fails when using flatten on JSON data where some documents have an empty array
> ------------------------------------------------------------------------------------
>
>                 Key: DRILL-3562
>                 URL: https://issues.apache.org/jira/browse/DRILL-3562
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - JSON
>    Affects Versions: 1.1.0
>            Reporter: Philip Deegan
>            Assignee: Serhii Harnyk
>              Labels: ready-to-commit
>             Fix For: Future
>
>
> Drill query fails when using flatten when some records contain an empty array 
> {noformat}
> SELECT COUNT(*) FROM (SELECT FLATTEN(t.a.b.c) AS c FROM dfs.`flat.json` t) flat WHERE
flat.c.d.e = 'f' limit 1;
> {noformat}
> Succeeds on 
> { "a": { "b": { "c": [  { "d": {  "e": "f" } } ] } } }
> Fails on
> { "a": { "b": { "c": [] } } }
> Error
> {noformat}
> Error: SYSTEM ERROR: ClassCastException: Cannot cast org.apache.drill.exec.vector.NullableIntVector
to org.apache.drill.exec.vector.complex.RepeatedValueVector
> {noformat}
> Is it possible to ignore the empty arrays, or do they need to be populated with dummy
data?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message