From user-return-515-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Fri Jun 19 04:10:57 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id BA00518065B for ; Fri, 19 Jun 2020 06:10:56 +0200 (CEST) Received: (qmail 7185 invoked by uid 500); 19 Jun 2020 04:10:55 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 7175 invoked by uid 99); 19 Jun 2020 04:10:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Jun 2020 04:10:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D4A95C05F5 for ; Fri, 19 Jun 2020 04:10:54 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.5 X-Spam-Level: X-Spam-Status: No, score=0.5 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id CzzQ6GoSW8DQ for ; Fri, 19 Jun 2020 04:10:53 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::52b; helo=mail-ed1-x52b.google.com; envelope-from=emkornfield@gmail.com; receiver= Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 06CF77D3FB for ; Fri, 19 Jun 2020 04:10:52 +0000 (UTC) Received: by mail-ed1-x52b.google.com with SMTP id o26so6542353edq.0 for ; Thu, 18 Jun 2020 21:10:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to; bh=8XiQPjPY1HUaV8AEPCnB2IGVHXgDYNoDYNfjPXeTBk4=; b=JaZyCTS9KBGVVT4T28Ogj0iSRrn92lKggGRdU1+z9tS+so3QUTWwgtV94f43Y3c0jc MyA/ZA4VvHo4qvyuPFbiwkQtmkHXzwEGZe2lusa4+nZWM65cAx9oeWDB9BMRg+hjoliL ohU5BBL86eKdmttBd+a707QNfbYQajMORFhGqB9m6J1UOPMcqOQLlMjPf57M5SdO06dn xkTruS1lrec8TSC6mYupIWbbcYAKGaoMs0bsEhykmO5nWg9wKFyDN2uVcEs0piajPo+y HXSHiI93BfW50XE65aB2KsDxjDX/EzB8xc2K7NZbRp800d1h29X2I3TBWdi2xLNiuqyH XiWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to; bh=8XiQPjPY1HUaV8AEPCnB2IGVHXgDYNoDYNfjPXeTBk4=; b=tR04jcuePYxTu3X6mQYmYqvFzHTKb8eNR0oN3C4MC+v9cU/KF/wdMTit16NKKCHRuL jiEBMTCi2LqQCXT08fKQ7iux1h1GbLTNZ8aswIMqPJCpc+9+Zpl+qJjj+q30a2aRuwe6 3267Nm93MwjCWFAfUSZrtiAVoxdaA6emH0XCdlPb3z+nphcjc7mBH3b51REiyVldeI0+ VsHvNpGqUB23kVijm7E3ki/OE+YNdK5CYwIiQnMFfb5EwJDAL7hTpP7p3kokaCqshOA/ 98p0tEiD2GHp2u0HRTeVITI2Q6RX6Ufvy5pOJ0xSGxGcBU/O7EQU7/tiUPFTWPtOnSVc Pnsw== X-Gm-Message-State: AOAM530Xse3jDSLcP+yWpg29ciBDUCT/lWxebFwf3MrVZwyBuTdKpfNC p370Umz9jcZX8XvwHLCvxco93vGJ6PGJLxKmrcHvzCtX X-Google-Smtp-Source: ABdhPJzyiBjgOUYLRKQ/PXg7agFgq6wilOXsNikICR5va/J/qfkHX1r3LH1okDnP1PmeY1Nt0dmT/tGxFlbDTVFgcY0= X-Received: by 2002:a05:6402:21c2:: with SMTP id bi2mr1383013edb.296.1592539852228; Thu, 18 Jun 2020 21:10:52 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: Reply-To: emkornfield@gmail.com From: Micah Kornfield Date: Thu, 18 Jun 2020 21:10:41 -0700 Message-ID: Subject: Re: Processing GetRecordBatcheReader PyArrow - 7800 To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="0000000000002ece4705a868147b" --0000000000002ece4705a868147b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Rollo, This is probably better discussed on dev@arrow.apache.org. In general, it is up to committers to hold the bar of what they think is appropriate to merge or not merge, so it is best to have the discussion directly with them= . Thanks, Micah On Fri, Jun 12, 2020 at 2:49 AM Rollo Konig-Brock wrote= : > Hey all, > > > > I started work on 7800, and have been running in a production system for = a > while now. I was a bit unclear on some API decisions, and sadly left the > branch to go stale. > > > > Then wjones1 also had a similar requirement, and forked off trying to > finish my work. This is also going stale. > > > > It seems like there are some really minor changes requested that I don=E2= =80=99t > think have any material effect, and some unclear question about the API > design (I=E2=80=99ve asked wjones1 for write access to his fork of my bra= nch). > > > > I think it=E2=80=99d be really great to get this in even if the API isn= =E2=80=99t the most > graceful thing. Right now the unpredictability memory usage can make > PyArrow unsuitable for a great number use cases. > > > > Sincerely, > > Rollo > > > --0000000000002ece4705a868147b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Rollo,
This is probably better discussed on dev@arrow.apache.org.=C2=A0 In general= , it is up to committers to hold the bar of what they think is appropriate = to merge or not merge, so it is best to have the discussion directly with t= hem.

Thanks,
Micah

On Fri, Jun 12, = 2020 at 2:49 AM Rollo Konig-Brock <= rollokb@gmail.com> wrote:

Hey al= l,

=C2=A0

I star= ted work on 7800, and have been running in a production system for a while = now. I was a bit unclear on some API decisions, and sadly left the branch t= o go stale.

=C2=A0

Then w= jones1 also had a similar requirement, and forked off trying to finish my w= ork. This is also going stale.

=C2=A0

It see= ms like there are some really minor changes requested that I don=E2=80=99t = think have any material effect, and some unclear question about the API des= ign (I=E2=80=99ve asked wjones1 for write access to his fork of my branch).

=C2=A0

I thin= k it=E2=80=99d be really great to get this in even if the API isn=E2=80=99t= the most graceful thing. Right now the unpredictability memory usage can m= ake PyArrow unsuitable for a great number use cases.

=C2=A0

Sincer= ely,

Rollo<= u>

=C2=A0

--0000000000002ece4705a868147b--