spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <>
Subject [jira] [Resolved] (SPARK-14536) NPE in JDBCRDD when array column contains nulls (postgresql)
Date Thu, 22 Sep 2016 09:08:20 GMT


Sean Owen resolved SPARK-14536.
    Resolution: Duplicate

> NPE in JDBCRDD when array column contains nulls (postgresql)
> ------------------------------------------------------------
>                 Key: SPARK-14536
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.6.1
>            Reporter: Jeremy Smith
>              Labels: NullPointerException
> At
it is assumed that the JDBC driver will definitely return a non-null `Array` object from the
call to `getArray`, and that in the event of a null array it will return an non-null `Array`
object with a null underlying array.  But as you can see here
that isn't the case, at least for PostgreSQL.  This causes a `NullPointerException` whenever
an array column contains null values. It seems like the PostgreSQL JDBC driver is probably
doing the wrong thing, but even so there should be a null check in JDBCRDD.  I'm happy to
submit a PR if that would be helpful.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message