drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-4479) JsonReader should pick a less restrictive type when creating the default column
Date Sat, 05 Mar 2016 01:40:40 GMT
Aman Sinha created DRILL-4479:
---------------------------------

             Summary: JsonReader should pick a less restrictive type when creating the default
column
                 Key: DRILL-4479
                 URL: https://issues.apache.org/jira/browse/DRILL-4479
             Project: Apache Drill
          Issue Type: Bug
          Components: Storage - JSON
    Affects Versions: 1.5.0
            Reporter: Aman Sinha


This JIRA is related to DRILL-3806 but has a narrower scope, so I decided to create separate
one. 

The JsonReader has the method ensureAtLeastOneField() (see https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java#L91)
that ensures that when no columns are found, create an empty one and it chooses to create
a nullable int column.  One consequence is that queries of the following type fail:

{noformat}
select c1 from dfs.`mostlynulls.json`;
...
...
| null  |
| null  |
Error: DATA_READ ERROR: Error parsing JSON - You tried to write a VarChar type when you are
using a ValueWriter of type NullableIntWriterImpl.

File  /Users/asinha/data/mostlynulls.json
Record  4097
{noformat}

In this file the first 4096 rows have NULL values for c1 followed by rows that have a valid
string.  

It would be useful for the Json reader to choose a less restrictive type such as varchar in
order to allow more types of queries to run.  





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message