From user-return-187-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Sat Aug 31 03:14:40 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 061F618065E for ; Sat, 31 Aug 2019 05:14:39 +0200 (CEST) Received: (qmail 1157 invoked by uid 500); 31 Aug 2019 03:14:38 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 1146 invoked by uid 99); 31 Aug 2019 03:14:38 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 31 Aug 2019 03:14:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id C43A0C2281 for ; Sat, 31 Aug 2019 03:14:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.801 X-Spam-Level: * X-Spam-Status: No, score=1.801 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 6SjMC6jpqieX for ; Sat, 31 Aug 2019 03:14:24 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.208.170; helo=mail-lj1-f170.google.com; envelope-from=emkornfield@gmail.com; receiver= Received: from mail-lj1-f170.google.com (mail-lj1-f170.google.com [209.85.208.170]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id B5352BC7A9 for ; Sat, 31 Aug 2019 03:14:23 +0000 (UTC) Received: by mail-lj1-f170.google.com with SMTP id l1so8115251lji.12 for ; Fri, 30 Aug 2019 20:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to; bh=0VHn8xinp5qUhGMBtpK32rwu/4y2P5q/pu5mnE/Mk7U=; b=mLkHgeYCzmu8lqK+tgxoTa8MViNTgx+DmSfUG8iDc6lEeDU+GdEKAb1RXK/JuxNI3x X1SVG1IuqnIpcZVMoG0/CTgxNdwf3MhUuV8Pwlx5KxLMdimat4/dBEY4W7sP1agq/VDi Wks/zmi0qYgGmfbIXhSkJkYUXzpEhOB6s5FMYkb7GslbD/xWtcgaFyNeXtk5tKj95eHu tyyFyfMBtdMnmX3xUfU23VHaUR3SBoPTRtLYGm+q2SI3PTNZQWwuEeIUb/IfHV1qJ93N SUs48dT0L56hasDPg363QnNWtu1Q5TMtpkw7t1Cvkuno3VaoLv7Kq98/3skos7XVSyJl 20Fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to; bh=0VHn8xinp5qUhGMBtpK32rwu/4y2P5q/pu5mnE/Mk7U=; b=qWLg1vI7fMnWD+ru8LDXzVQflsDKw9BxxEAKqvZ8hbnUzSpkDMYaYFB2KS8Q1qGuJh AtgsAqoXpZFq+ZcvZNaN6ggohHYT0mLKM4jMxQNqD7nL1KPuEuKugHCjAS0+asIeUWb0 Rs5pOXujcG93myJEitoxEiXei4R77FFiox5oNRRo7IPNIBo8nR573EKnLtSkiWz3Nw9K 1V0aHg7OH/J4x6FojIP7M9W3Cwr8gPLWqZCX8dNCoTxHCZBTVfVfM0NJDpdjhswl+FTq LGQ38lYZcobT3tiqSdGGDj8tZFXPFluAcy0i11nqmiIT8O5EZY8FQbIUzVBzXvLxZGwM ye6g== X-Gm-Message-State: APjAAAXhtbBM+00Iz8jUvkuSA/DdQolMvIZA+cMswIVH7778mkKJXOUZ AznMFXCyvY+9inmiczHeW8K0YWkudpGpkobJ3RAbgQ7F X-Google-Smtp-Source: APXvYqy/aCPWjP4MHHYY+XjHsrehSeCrTZrwsR/ypqoaoaBTklFg87Bt0q6vVlEYuFugJr6Spv9QZkeNPPsN4nZoBjk= X-Received: by 2002:a2e:a202:: with SMTP id h2mr10270983ljm.146.1567221255947; Fri, 30 Aug 2019 20:14:15 -0700 (PDT) MIME-Version: 1.0 References: <16ce1f3a1c0.27b7.727fde9afb6860196ef19c74546e60d2@ipp.mpg.de> <16ce3e95f10.27b7.727fde9afb6860196ef19c74546e60d2@ipp.mpg.de> In-Reply-To: <16ce3e95f10.27b7.727fde9afb6860196ef19c74546e60d2@ipp.mpg.de> Reply-To: emkornfield@gmail.com From: Micah Kornfield Date: Fri, 30 Aug 2019 20:14:04 -0700 Message-ID: Subject: Re: Reccord-Level Access To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="0000000000003eca1f0591612260" --0000000000003eca1f0591612260 Content-Type: text/plain; charset="UTF-8" Hi Simon, A couple notes: - Scalars are a C++ thing, There are ValueHolders in Java but i'm not sure what you want. - Dremio has a JDBC adaptor [1] that might be worth looking at (or maybe porting pieces of it into Arrow). Thanks, Micah [1] https://github.com/dremio/dremio-oss/blob/04e0387d474f1408731da0029aef7ecfad5e4d08/client/jdbc/src/main/java/com/dremio/jdbc/impl/DremioResultSetImpl.java On Fri, Aug 30, 2019 at 12:03 PM Simon Dumke wrote: > Hi Ben, > > thanks for the suggestion, i'll loon into it! > > Regards, > Simon > > Am 30. August 2019 20:59:23 schrieb Ben Kietzman >: > >> Hi Simon, >> >> If you're interested in adding a record interface, the Scalar classes >> might be a good place to start. They represent a value from an array slot >> and it should be fairly straightforward to extract a table row as a >> StructScalar >> >> On Fri, Aug 30, 2019 at 1:27 PM Wes McKinney wrote: >> >>> hi Simon -- I don't think there is any such Row accessor class in Java >>> but you are welcome to contribute one to the project. For performance >>> sensitive applications, using a record interface might not be the best >>> idea, but I can understand the convenience for some uses cases. >>> >>> - Wes >>> >>> On Fri, Aug 30, 2019 at 4:55 AM Simon Dumke >>> wrote: >>> > >>> > Hi all, >>> > >>> > >>> > >>> > >>> > I did not find anything (and so: no definite answer) in the docs, so i >>> > thought to ask here: >>> > >>> > >>> > >>> > >>> > Does Arrow (and at this point my main concern is Arrow for java) >>> support >>> > any type of concept that allows a "record level access" (so, a "row") >>> to >>> > data in an Arrow RecordBatch or Table? I would have thougt that even in >>> > column-oriented analytics etc. this would be a common last step access >>> > pattern over many use cases, but i could not find any references to >>> such a >>> > thing. >>> > >>> > >>> > >>> > >>> > Thanks and kind regards, >>> > Simon >>> > >>> > >>> >> > --0000000000003eca1f0591612260 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Simon,
A couple notes:
- Scalars ar= e a C++ thing, There are ValueHolders in Java but i'm not sure what you= want.
- Dremio has a JDBC adaptor [1] that might be worth looking at (= or maybe porting pieces of it into Arrow).

On Fri, Aug 30, 2019 at 12:= 03 PM Simon Dumke <simon.dumke@ipp.mpg.de> wrote:
Hi Ben,

thanks=C2=A0for the suggestion, i'll l= oon into it!
<= br>
Regards,
Simon

Am 30. August 2019 20:59:23 schrieb Ben Kietzman <ben.kietzman@rstudio.com>= :

Hi Simon,

If you're inte= rested in adding a record interface, the Scalar classes might be a good pla= ce to start. They represent a value from an array slot and it should be fai= rly straightforward to extract a table row as a StructScalar

On Fr= i, Aug 30, 2019 at 1:27 PM Wes McKinney <wesmckinn@gmail.com> wrote:
hi Simon -- I don't think = there is any such Row accessor class in Java
but you are welcome to contribute one to the project. For performance
sensitive applications, using a record interface might not be the best
idea, but I can understand the convenience for some uses cases.

- Wes

On Fri, Aug 30, 2019 at 4:55 AM Simon Dumke <simon.dumke@ipp.mpg.de> wrote:
>
> Hi all,
>
>
>
>
> I did not find anything (and so: no definite answer) in the docs, so i=
> thought to ask here:
>
>
>
>
> Does Arrow (and at this point my main concern is Arrow for java) suppo= rt
> any type of concept that allows a "record level access" (so,= a "row") to
> data in an Arrow RecordBatch or Table? I would have thougt that even i= n
> column-oriented analytics etc. this would be a common last step access=
> pattern over many use cases, but i could not find any references to su= ch a
> thing.
>
>
>
>
> Thanks and kind regards,
> Simon
>
>

--0000000000003eca1f0591612260--