From user-return-1212-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Tue May 4 10:39:28 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 324ED18065E for ; Tue, 4 May 2021 12:39:28 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 21E923F944 for ; Tue, 4 May 2021 10:39:13 +0000 (UTC) Received: (qmail 79376 invoked by uid 500); 4 May 2021 10:39:12 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 79366 invoked by uid 99); 4 May 2021 10:39:12 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 May 2021 10:39:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id CB9AB1FF49B for ; Tue, 4 May 2021 10:39:11 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id Q5Cs3IRmtlx4 for ; Tue, 4 May 2021 10:39:11 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::134; helo=mail-lf1-x134.google.com; envelope-from=joris.mg.peeters@gmail.com; receiver= Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 556387FD9C for ; Tue, 4 May 2021 10:39:11 +0000 (UTC) Received: by mail-lf1-x134.google.com with SMTP id 12so12512267lfq.13 for ; Tue, 04 May 2021 03:39:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=pw+341ZTE9nqvA1Z4/13VvhRLXLtdw9zmY9pbT0Lwgc=; b=A0J3TPKbfSErgwoziV/ZujC4M+tD3bVi/wJrW9dcpOug9fFzY5ydR5Ii8L+tNN0bkY zuaQZ8HymCIZhERci7+7pSZXRnMTL27sjiGNv5rv5ohpc71YFjTEia7FLs/EY1iwVe/A Euj+FCffM/v/eoGCUbY4ixz5y2v2AnFi5KubCqmnBErbe4knFdoK20HJrrT4icWDQ5TY 2OonC6OwnxEdPXCBavEojARTdZCvDFDJHcMHcSMgt0JrUc0mppuU+DV5Q4XGgHxIWLRm +p0mfI8J0eu9Z2dNpqT7VflNO7QdKdgpb7dgcQZNvDPqK2mEjSu7NjUuQOOUPNdwplhY pEug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=pw+341ZTE9nqvA1Z4/13VvhRLXLtdw9zmY9pbT0Lwgc=; b=uNxa/B9T2q8hKEfeLXCTjfy8SkZTJu8pJ/T0g1fldyiwf+qZ6oRk68VLrd7fH3o0uB JSvF5kiUrXRma7KVRNWk6jjne+bSvyq0iMYdM3tFmSjmV5M65wbExJi0SEd1flqSywUr R6Rl7HT698tdPXM0FooyXOdvCtTjvmIbsBYKMWK74YnmFEgBzUHtHiHPE1wxHbNeOv7z hqfifSttOPf5PyauhnXTwgBm4NIlwVeVUwE0jFEu3sFIdCpr4/6UngA41yi0ZVQz0Hxs n5QXH6be8iIsX7KECFLziPwkAU+vGIddVtPX4oZmZwtupyPWfH5adfOxOmblxL0oZiW1 DgrQ== X-Gm-Message-State: AOAM531ioL3c7vhm1Nwf1GL+z+5H0WTz0PSV63aw+N6lkS1ibnXyLKkd eV4q08Je72sD5YgqtBd+YHc+kjUH+lh4K1uLdae3hht/H5Y= X-Google-Smtp-Source: ABdhPJxxlXktTmamIR091+UdQPVYebkTRR+xASV8pgbTEtjB5o5tUSC8+XXLf+T7Axmi95qbey4WCfFpAtCFC8aGIz8= X-Received: by 2002:a05:6512:230b:: with SMTP id o11mr1267280lfu.93.1620124750678; Tue, 04 May 2021 03:39:10 -0700 (PDT) MIME-Version: 1.0 From: Joris Peeters Date: Tue, 4 May 2021 11:38:59 +0100 Message-ID: Subject: [Java JDBC adapter] non-nullable fields? To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="00000000000041904505c17eb038" --00000000000041904505c17eb038 Content-Type: text/plain; charset="UTF-8" I'm looking to use the Java JDBC adapter for loading tables from SQL Server into Arrow record batches. At first glance the Arrow JDBC adapter seems to work well but, unless I'm mistaken, it simply makes every vector nullable, irrespective of whether the corresponding SQL column is nullable or not. I think the line final FieldType fieldType = new FieldType(true, arrowType, /* dictionary encoding */ null, metadata); in https://github.com/apache/arrow/blob/master/java/adapter/jdbc/src/main/java/org/apache/arrow/adapter/jdbc/JdbcToArrowUtils.java#L158 might be the cause here. Is my interpretation correct, or am I missing a setting of sorts? If indeed correct, is there a fundamental reason the NULL-ness is not transferred, or is this something I could contribute in a PR? (which I'd be happy to) I guess it's just a matter of inspecting the result metadata. Cheers, -J --00000000000041904505c17eb038 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I'm looking to use the Java JDBC adapter for loading t= ables from SQL Server into Arrow record batches.=C2=A0

A= t first glance the Arrow JDBC adapter seems to work well but, unless I'= m mistaken, it simply makes every vector nullable, irrespective of whether = the corresponding SQL column is nullable or not.=C2=A0

=
I think the line=C2=A0

final FieldType fieldT= ype =3D new FieldType(true, arrowType, /* dictionary encoding */ null, meta= data);


Is m= y interpretation correct, or am I missing a setting of sorts? If indeed cor= rect, is there a fundamental reason the NULL-ness is not transferred, or is= this something I could contribute in a PR? (which I'd be happy to) I g= uess it's just a matter of inspecting the result metadata.
Cheers,
-J
--00000000000041904505c17eb038--