From user-return-188-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Sat Aug 31 09:29:42 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4E64018063F for ; Sat, 31 Aug 2019 11:29:42 +0200 (CEST) Received: (qmail 28470 invoked by uid 500); 31 Aug 2019 09:29:41 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 28452 invoked by uid 99); 31 Aug 2019 09:29:41 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 31 Aug 2019 09:29:41 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id A6A9BC0BFF for ; Sat, 31 Aug 2019 09:29:40 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.703 X-Spam-Level: X-Spam-Status: No, score=0.703 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id RbUUpiFy2J7Z for ; Sat, 31 Aug 2019 09:29:37 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=194.95.238.160; helo=c1962.mx.srv.dfn.de; envelope-from=simon.dumke@ipp.mpg.de; receiver= Received: from c1962.mx.srv.dfn.de (c1962.mx.srv.dfn.de [194.95.238.160]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id E4BF07D3FC for ; Sat, 31 Aug 2019 09:29:36 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by c1962.mx.srv.dfn.de (Postfix) with ESMTP id 626B120005A for ; Sat, 31 Aug 2019 11:29:36 +0200 (CEST) Received: from c1962.mx.srv.dfn.de ([127.0.0.1]) by localhost (mgw4-erl.srv.dfn.de [127.0.0.1]) (amavisd-new, port 20123) with ESMTP id P-TF01J5lRwj for ; Sat, 31 Aug 2019 11:29:35 +0200 (CEST) Received: from post.rzg.mpg.de (post.rzg.mpg.de [130.183.30.42]) by c1962.mx.srv.dfn.de (Postfix) with ESMTPS for ; Sat, 31 Aug 2019 11:29:35 +0200 (CEST) Received: from [192.168.0.118] (x4d039b93.dyn.telefonica.de [77.3.155.147]) (authenticated bits=0) by post.rzg.mpg.de (8.14.7/8.14.7) with ESMTP id x7V9TYio018434 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sat, 31 Aug 2019 11:29:35 +0200 From: Simon Dumke To: Date: Sat, 31 Aug 2019 11:29:33 +0200 Message-ID: <16ce7022448.27b7.727fde9afb6860196ef19c74546e60d2@ipp.mpg.de> In-Reply-To: References: <16ce1f3a1c0.27b7.727fde9afb6860196ef19c74546e60d2@ipp.mpg.de> <16ce3e95f10.27b7.727fde9afb6860196ef19c74546e60d2@ipp.mpg.de> User-Agent: AquaMail/1.20.0-1469 (build: 102100004) Subject: Re: Reccord-Level Access MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----------16ce7022705757e27b783ad657" This is a multi-part message in MIME format. ------------16ce7022705757e27b783ad657 Content-Type: text/plain; format=flowed; charset="us-ascii" Content-Transfer-Encoding: 8bit Hi Micah, thanks for pointing this out, when first reading Bens Message I misread "the Scalar classes" as pointing to an Arrow Implementation in the Scalar language (and assuming a record construct existing there), not a set of Java (or rather C++) classes. About dremio's jdbc: this was exactly what i was looking for when thinking of arrow in an sql (like) context. Thanks a lot! Regards, Simon Am 31. August 2019 05:14:42 schrieb Micah Kornfield : > Hi Simon, > A couple notes: > - Scalars are a C++ thing, There are ValueHolders in Java but i'm not sure > what you want. > - Dremio has a JDBC adaptor [1] that might be worth looking at (or maybe > porting pieces of it into Arrow). > > Thanks, > Micah > > [1] > https://github.com/dremio/dremio-oss/blob/04e0387d474f1408731da0029aef7ecfad5e4d08/client/jdbc/src/main/java/com/dremio/jdbc/impl/DremioResultSetImpl.java > > > > On Fri, Aug 30, 2019 at 12:03 PM Simon Dumke wrote: > Hi Ben, > > thanks for the suggestion, i'll loon into it! > > > Regards, > Simon > > Am 30. August 2019 20:59:23 schrieb Ben Kietzman : >> Hi Simon, >> >> If you're interested in adding a record interface, the Scalar classes might >> be a good place to start. They represent a value from an array slot and it >> should be fairly straightforward to extract a table row as a StructScalar >> >> On Fri, Aug 30, 2019 at 1:27 PM Wes McKinney wrote: >> hi Simon -- I don't think there is any such Row accessor class in Java >> but you are welcome to contribute one to the project. For performance >> sensitive applications, using a record interface might not be the best >> idea, but I can understand the convenience for some uses cases. >> >> - Wes >> >> On Fri, Aug 30, 2019 at 4:55 AM Simon Dumke wrote: >>> >>> Hi all, >>> >>> >>> >>> >>> I did not find anything (and so: no definite answer) in the docs, so i >>> thought to ask here: >>> >>> >>> >>> >>> Does Arrow (and at this point my main concern is Arrow for java) support >>> any type of concept that allows a "record level access" (so, a "row") to >>> data in an Arrow RecordBatch or Table? I would have thougt that even in >>> column-oriented analytics etc. this would be a common last step access >>> pattern over many use cases, but i could not find any references to such a >>> thing. >>> >>> >>> >>> >>> Thanks and kind regards, >>> Simon >>> >>> ------------16ce7022705757e27b783ad657 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Hi Micah,

thanks for pointing this out, when first reading Bens Message I misread= "the Scalar classes" as pointing to an Arrow Implementation in the Scalar = language (and assuming a record construct existing there), not a set of Jav= a (or rather C++) classes.

About dremio's jdbc: this was exactly what i was looking for when thinki= ng of arrow in an sql (like) context. Thanks a lot!
=
Regards,
Simon

Am 31. August 2019 05:14:42 schrieb Micah Kornfield <emkornfiel= d@gmail.com>:

Hi Simon,
A couple notes:
- Scalars ar= e a C++ thing, There are ValueHolders in Java but i'm not sure what you wan= t.
- Dremio has a JDBC adaptor [1] that might be worth looking at (or m= aybe porting pieces of it into Arrow).
On Fri, Aug 30, 2019 at 12:03 P= M Simon Dumke <simon.dumke@ipp.mpg.de> wrote:
Hi Ben,

thanks for the suggestion, i'll loon = into it!

<= /span>
Regards,
Simon
<= div dir=3D"auto">

Am 30. August 2019 20:59:23 schrieb Ben Kietzman <ben.kietzman@rstudio.com>= :

Hi Simon,

If you're interest= ed in adding a record interface, the Scalar classes might be a good place t= o start. They represent a value from an array slot and it should be fairly = straightforward to extract a table row as a StructScalar
On Fri, A= ug 30, 2019 at 1:27 PM Wes McKinney <wesmckinn@gmail.com> wrote:
hi Simon -- I don't think there is= any such Row accessor class in Java
but you are welcome to contribute one to the project. For performance
sensitive applications, using a record interface might not be the best
idea, but I can understand the convenience for some uses cases.

- Wes

On Fri, Aug 30, 2019 at 4:55 AM Simon Dumke <simon.dumke@ipp.mpg.de> wrote:
>
> Hi all,
>
>
>
>
> I did not find anything (and so: no definite answer) in the docs, so i=
> thought to ask here:
>
>
>
>
> Does Arrow (and at this point my main concern is Arrow for java) suppo= rt
> any type of concept that allows a "record level access" (so, a "row") = to
> data in an Arrow RecordBatch or Table? I would have thougt that even i= n
> column-oriented analytics etc. this would be a common last step access=
> pattern over many use cases, but i could not find any references to su= ch a
> thing.
>
>
>
>
> Thanks and kind regards,
> Simon
>
>


------------16ce7022705757e27b783ad657--