From user-return-1046-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Tue Mar 2 17:08:16 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-he-de.apache.org (mxout1-he-de.apache.org [95.216.194.37]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 4AB99180630 for ; Tue, 2 Mar 2021 18:08:16 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-he-de.apache.org (ASF Mail Server at mxout1-he-de.apache.org) with SMTP id 08C3664506 for ; Tue, 2 Mar 2021 17:08:14 +0000 (UTC) Received: (qmail 18668 invoked by uid 500); 2 Mar 2021 17:08:14 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 18658 invoked by uid 99); 2 Mar 2021 17:08:14 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2021 17:08:14 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id 771E11FF39A for ; Tue, 2 Mar 2021 17:08:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id VS01DtBcBiiD for ; Tue, 2 Mar 2021 17:08:13 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::834; helo=mail-qt1-x834.google.com; envelope-from=prmittal.nch@gmail.com; receiver= Received: from mail-qt1-x834.google.com (mail-qt1-x834.google.com [IPv6:2607:f8b0:4864:20::834]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id DF1AA7FCC3 for ; Tue, 2 Mar 2021 17:08:12 +0000 (UTC) Received: by mail-qt1-x834.google.com with SMTP id w1so15302001qto.2 for ; Tue, 02 Mar 2021 09:08:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=1XyIbZTgHO1+Owz+N+5cIRHruo27NB6aYEiJmYuE39s=; b=l+2/C5hXUFhAGBI6tf9hZyy4qoIrrL8yTNn+76TgA4jbUZnTDwuNTOO0ZHY9b0cFMe t2o62y5rBk5LFAwBYOHf2Ej+Oq8ySOiZKsLsewplAOvciNegkb6OV4Ex/4/PrJlYUGz+ TMd+Jee2nlqDbXuHSbAFmBLeq6K8eQyv3HRNQG8xrct6gsfU3zMV3XaCj7+5fCqhbYsc YcQgH2cHPIa0uoWbLVyKY2SuYnF4+Iwqfk1ilX2Gt87tlbwk0L0pOUgwBQw6GOS4c3Ii PpvKj+ZDZqTEpIAVZK6ATPkFe+1/mUkKGixWoZ1pOmh2LVVvXN9eGtTTPtNQAVLUsm8i jTzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=1XyIbZTgHO1+Owz+N+5cIRHruo27NB6aYEiJmYuE39s=; b=e7slC9TbAyq2qYsuYIjjxEiQ397cquKfIwoRRAV6pgTZVbYkQimF9xx976pTuATRUs vu+izU/WqpRGXf3VVMwLWz15cojo9m8Fn3grBdei6ISrLX92FbD5PdUYWMtHrXdv7B71 3ALb8flT+rMGUapt9tSjH1uOL3jyz9uPg9Y5beIMCxPwdqH6UFsPukG2qtAzLEJUmdz/ ObwHOZYv0kc2r3bkM3D8lMLHpUCZZVLZ2YSGP0ancFVBlxdJkmgfAwOhAHoEWRamQFTn ImIc/81k/ezeHADrIp25iWMxpmQ2TM3130BuMAu39D+gpHVjNXqD/7A41JlR4lraoDC4 bK/A== X-Gm-Message-State: AOAM531TYg0relIjRvcGZTeIAZWzR225LN5PyXXs8nmcPoSRUBCDfsjX iB6ho4uUxhNJxoeaiPHeAnaBzzaH9sLvEbtdfLW9dqYI X-Google-Smtp-Source: ABdhPJyfs2FsyKqQYMN/1Y8KbELBOvE3YTjoT/gR2xD01ZqRs3ePCGm8gENhJJV/AjW3Ph8oGxi+pNVTFqPxluSbGYA= X-Received: by 2002:ac8:5747:: with SMTP id 7mr19052150qtx.274.1614704890991; Tue, 02 Mar 2021 09:08:10 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Priyanshu Mittal Date: Tue, 2 Mar 2021 22:37:57 +0530 Message-ID: Subject: Re: How to connect arrow flights to spark clusters To: user@arrow.apache.org, emkornfield@gmail.com Content-Type: multipart/alternative; boundary="00000000000071d8cb05bc90c768" --00000000000071d8cb05bc90c768 Content-Type: text/plain; charset="UTF-8" One more query here, Although I'm still not able to figure out how to build and run the code in this repo https://github.com/rymurr/flight-spark-source but hopefully if it works, then I would like to know that is the code in this repo for flight spark connector is some sample code written by an individual or it is official spark flight connector code supported by arrow or some other community? Thanks, Priyanshu On Tue, 2 Mar, 2021, 8:52 am Priyanshu Mittal, wrote: > Hi Micah, > > I tried to understand this repo > https://github.com/rymurr/flight-spark-source but I'm not able to build > and run this code. Readme file also doesn't explain how to run this code.. > So, can anyone help me in running this code? And also is there any > documentation related to it? > > Further, I found very less resources for usage of Arrow flight and like > how we can use it in our project. So If there are any implementations of > Arrow flight or Spark Flight connector, It would be great if you can share > that with us. > > Thanks, > Priyanshu > > On Sun, Feb 28, 2021 at 10:40 PM Priyanshu Mittal > wrote: > >> Hi, >> Thanks for sharing that. I'm looking into it. >> >> On Sat, 27 Feb, 2021, 10:21 am Micah Kornfield, >> wrote: >> >>> Ryan Murray prototyped something like this a while ago [1]. I'm not >>> sure what the current state of this is. >>> >>> [1] https://github.com/rymurr/flight-spark-source >>> >>> On Thu, Feb 18, 2021 at 10:15 PM Priyanshu Mittal < >>> prmittal.nch@gmail.com> wrote: >>> >>>> Hi, >>>> I want to connect my arrow flight with apache spark clusters so that I >>>> can transfer my data between spark clusters and arrow flights. Basically, I >>>> have to transfer my arrow data to spark clusters using arrow flights. So, >>>> How can I do that? >>>> Any help or resources would be appreciated. >>>> >>>> Thanks, >>>> Priyanshu >>>> >>> --00000000000071d8cb05bc90c768 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
One more query here,
Although I'm st= ill not able to figure out how to build and run the code in this repo https://github.com/rym= urr/flight-spark-source but hopefully if it works, then I would like to= know that is the code in this repo for flight spark connector is some samp= le code written by an individual or it is official spark flight connector c= ode supported by arrow or some other community?

=
Thanks,
Priyanshu

On Tu= e, 2 Mar, 2021, 8:52 am Priyanshu Mittal, <prmittal.nch@gmail.com> wrote:
Hi Micah,

I tried to un= derstand this repo=C2=A0https://github.com/rymurr/fligh= t-spark-source=C2=A0but I'm not able to build and run this code. Re= adme file also doesn't explain how to run this code.. So, can anyone he= lp me in running this code? And also is there any documentation related to = it?

Further, I found very less resources for usage= of Arrow flight and like how we can use it in our project. So If there are= any implementations of Arrow flight or Spark Flight connector, It would be= great if you can share that with us.

Thanks,
Priyanshu

On Sun, Feb 28, 2021 at 10:40 PM Priyanshu Mittal <= prmittal.nch@gmail.com> wrote:
Hi,
Thanks for= sharing that. I'm looking into it.=C2=A0

On Sat, 27 Feb, 2021, 10= :21 am Micah Kornfield, <emkornfield@gmail.com> wro= te:
Ryan Murray prototyped something like this a while ago [1].=C2=A0 I= 9;m not sure what the current state of this is.


On Thu, Feb 18, 2021 at 10:15 PM Priyanshu Mittal = <prmittal.nch@gmail.com> wrote:
Hi,I want to connect my arrow flight with apache spark cluster= s so that I can transfer my data between spark clusters and arrow flights. = Basically, I have to transfer my arrow data to spark clusters using arrow f= lights. So, How can I do that?
Any help or resources= would be appreciated.

T= hanks,
Priyanshu
--00000000000071d8cb05bc90c768--