drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4479) JsonReader should pick a less restrictive type when creating the default column
Date Wed, 09 Mar 2016 07:45:41 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15186687#comment-15186687
] 

ASF GitHub Bot commented on DRILL-4479:
---------------------------------------

GitHub user amansinha100 opened a pull request:

    https://github.com/apache/drill/pull/420

    DRILL-4479: Use varchar for default column when all_text_mode is enab…

    …led.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/amansinha100/incubator-drill DRILL-4479

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/drill/pull/420.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #420
    
----
commit c5b4aef5b35547561ea71ce880391429643a6ee0
Author: Aman Sinha <asinha@maprtech.com>
Date:   2016-03-08T17:27:32Z

    DRILL-4479: Use varchar for default column when all_text_mode is enabled.

----


> JsonReader should pick a less restrictive type when creating the default column
> -------------------------------------------------------------------------------
>
>                 Key: DRILL-4479
>                 URL: https://issues.apache.org/jira/browse/DRILL-4479
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - JSON
>    Affects Versions: 1.5.0
>            Reporter: Aman Sinha
>         Attachments: mostlynulls.json
>
>
> This JIRA is related to DRILL-3806 but has a narrower scope, so I decided to create separate
one. 
> The JsonReader has the method ensureAtLeastOneField() (see https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java#L91)
that ensures that when no columns are found, create an empty one and it chooses to create
a nullable int column.  One consequence is that queries of the following type fail:
> {noformat}
> select c1 from dfs.`mostlynulls.json`;
> ...
> ...
> | null  |
> | null  |
> Error: DATA_READ ERROR: Error parsing JSON - You tried to write a VarChar type when you
are using a ValueWriter of type NullableIntWriterImpl.
> File  /Users/asinha/data/mostlynulls.json
> Record  4097
> {noformat}
> In this file the first 4096 rows have NULL values for c1 followed by rows that have a
valid string.  
> It would be useful for the Json reader to choose a less restrictive type such as varchar
in order to allow more types of queries to run.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message