drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Altekruse (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-1728) Better error messages on Drill JSON read error
Date Tue, 25 Nov 2014 01:40:13 GMT

    [ https://issues.apache.org/jira/browse/DRILL-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223913#comment-14223913

Jason Altekruse commented on DRILL-1728:

I think this fix could happen in two places. The message is coming out of the Drill Field/Map
Writer interfaces that allow us to easily read and write complex data. I think that these
need to produce user consumable messages, as there are other parts of the Drill code (such
as the complex parquet reader, covertFromJSON, kvgen, etc.) that use this interface and will
fail in the same manner currently.

On top of this fix, it would be useful if the various consumers of this interface, like the
json reader would provide additional context information on top of the low level (but soon
to be more readable) error. For the JSON reader it would be nice if this could include a filename
and line number and field in the JSON document where the error occurred. For components later
in execution this becomes less straight forward, as the data streaming in from several different
incoming operators will come in a random order. It might be possible to include whatever information
we have about the current or last successfully handled record in the error message to help
users identify the issue in the dataset.

There is also the outstanding patch for just inserting null in the dataset when there is an
error. [~mehant] I could not find the issue number for this, could you post it here? I think
this needs to be merged, and if it does not return empty list or empty map appropriately when
we have an error operating on a complex type, it should be enhanced to give the same flexibility
if the users need to interact with imperfect complex data.

> Better error messages on Drill JSON read error
> ----------------------------------------------
>                 Key: DRILL-1728
>                 URL: https://issues.apache.org/jira/browse/DRILL-1728
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Tomer Shiran
>            Assignee: Jason Altekruse
>             Fix For: 0.7.0
> {code}
> 0: jdbc:drill:zk=localhost:2181> SELECT * FROM dfs.root.`Users/tshiran/Development/demo/data/yelp/business.json`
WHERE true and REPEATED_CONTAINS(categories, 'Australian');
> +-------------+--------------+------------+------------+------------+------------+--------------+------------+------------+------------+------------+------------+------------+------------+---------------+
> | business_id | full_address |   hours    |    open    | categories |    city    | review_count
|    name    | longitude  |   state    |   stars    |  latitude  | attributes |    type  
 | neighborhoods |
> +-------------+--------------+------------+------------+------------+------------+--------------+------------+------------+------------+------------+------------+------------+------------+---------------+
> Query failed: Query stopeed., You tried to start when you are using a ValueWriter of
type NullableBitWriterImpl. [ e5bafa1e-6226-443d-80fd-51e18f330899 on ]
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing query.
> 	at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> 	at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> 	at sqlline.SqlLine.print(SqlLine.java:1809)
> 	at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> 	at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> 	at sqlline.SqlLine.dispatch(SqlLine.java:889)
> 	at sqlline.SqlLine.begin(SqlLine.java:763)
> 	at sqlline.SqlLine.start(SqlLine.java:498)
> 	at sqlline.SqlLine.main(SqlLine.java:460)
> {code}

This message was sent by Atlassian JIRA

View raw message