arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fan Liya <liya.fa...@gmail.com>
Subject Re: [Java JDBC adapter] non-nullable fields?
Date Thu, 06 May 2021 04:52:47 GMT
Hi Joris,

Thanks for reporting the problem.

We make use of the nullable information in ArrowVectorIterator#initialize.
(Details can be found in
https://github.com/apache/arrow/blob/master/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/ArrowVectorIterator.java#L73
)

Please note that the  ArrowVectorIterator is our encouraged way of using
the JDBC adapter.

Best,
Liya Fan


On Wed, May 5, 2021 at 1:42 PM Micah Kornfield <emkornfield@gmail.com>
wrote:

> I would need to look further, but I thought we handled null vs not null.
> At least I thought we had specialized conversion code to avoid branches.
> If this isn't the case it seems reasonable to contribute a path.
>
> On Tue, May 4, 2021 at 3:39 AM Joris Peeters <joris.mg.peeters@gmail.com>
> wrote:
>
>> I'm looking to use the Java JDBC adapter for loading tables from SQL
>> Server into Arrow record batches.
>>
>> At first glance the Arrow JDBC adapter seems to work well but, unless I'm
>> mistaken, it simply makes every vector nullable, irrespective of whether
>> the corresponding SQL column is nullable or not.
>>
>> I think the line
>>
>> final FieldType fieldType = new FieldType(true, arrowType, /* dictionary
>> encoding */ null, metadata);
>>
>> in
>> https://github.com/apache/arrow/blob/master/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrowUtils.java#L158
>> might be the cause here.
>>
>> Is my interpretation correct, or am I missing a setting of sorts? If
>> indeed correct, is there a fundamental reason the NULL-ness is not
>> transferred, or is this something I could contribute in a PR? (which I'd be
>> happy to) I guess it's just a matter of inspecting the result metadata.
>>
>> Cheers,
>> -J
>>
>

Mime
View raw message