From user-return-1038-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Tue Mar 2 03:22:23 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 501FE18062C for ; Tue, 2 Mar 2021 04:22:23 +0100 (CET) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 8D860431B2 for ; Tue, 2 Mar 2021 03:22:22 +0000 (UTC) Received: (qmail 50715 invoked by uid 500); 2 Mar 2021 03:22:20 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 50612 invoked by uid 99); 2 Mar 2021 03:22:19 -0000 Received: from spamproc1-he-fi.apache.org (HELO spamproc1-he-fi.apache.org) (95.217.134.168) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Mar 2021 03:22:19 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-fi.apache.org (ASF Mail Server at spamproc1-he-fi.apache.org) with ESMTP id 64EFEC033E for ; Tue, 2 Mar 2021 03:22:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-fi.apache.org X-Spam-Flag: NO X-Spam-Score: -0.002 X-Spam-Level: X-Spam-Status: No, score=-0.002 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamproc1-he-fi.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-ec2-va.apache.org ([116.203.227.195]) by localhost (spamproc1-he-fi.apache.org [95.217.134.168]) (amavisd-new, port 10024) with ESMTP id j8EaH0AtBxtR for ; Tue, 2 Mar 2021 03:22:17 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.221.54; helo=mail-wr1-f54.google.com; envelope-from=prmittal.nch@gmail.com; receiver= Received: from mail-wr1-f54.google.com (mail-wr1-f54.google.com [209.85.221.54]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id 329DCBCF9F for ; Tue, 2 Mar 2021 03:22:17 +0000 (UTC) Received: by mail-wr1-f54.google.com with SMTP id 7so18329236wrz.0 for ; Mon, 01 Mar 2021 19:22:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=0onxlu7dTKbgJpiu6dRdhGGSSPdu1Tsin5XPsiawH/8=; b=UMeqVzmeGOpnHGe6g/7G9fn8doXoRIgY0FydqSfGx+6ZXXjMrhcGSBLTXKxz5n8MSe KcgdsxKOe6zjtVA8bo4QhwLUZ5RnRa0aQwXExCY77w9yjWOvI/iLy6AQNmIg7MA5Pjzn PhM+D7jtCIeIoTj+nOPsMklgQ91NmzHddhi/huaJIV9ewHmcOxw+JjCPy9A96BeZZEs6 fOW+noAIFjKUnH9zxVKNET4wvnuYYF9RyKBinJfd3oSNXIxGJ9YGVU1dKHxQJBaZZdsA EElR0bVAm5GzRq2UX5FGrS2pkdAp1zERS5UI+NJwQlj/d3i9RHGZI6ZRXp8ehO3Ghab5 y18A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=0onxlu7dTKbgJpiu6dRdhGGSSPdu1Tsin5XPsiawH/8=; b=Gkz5IiJzk1Ww6p0ew+maMcKdKmjLB3m7kXoQagP6HOGHBnWrP2OKSK80JKfgwqBQjo hoYyu3mO4FuvkPUZ6AcYeHRbnwUn6ca5T+ySgOtiT3bQWEhGi5jsHhGqWUy2YWRgSIQM DuTdFResq2Gagr9oPGRl2asE7JZ88AoF7vP/sw/WjXNRcR68L/ibQKdd0RdH9VGjK7VP iV/dKA2cm6X67AUTUohNMr8SGn722zHnZuydavNSlKaAoWThrd2315fCQjbZ3xOdWsZw GD9AI/q9nRdAavFo4SuFz37cvNXKYZmLbw/gCHJ4eG4PFHYA6264SfF+ENIW+3oYU+/Q M8jw== X-Gm-Message-State: AOAM5300EgQ2IM+DkvjW/butFQzNL0kDXDMAHRy914WzNNpl4cG+DPdc ZpICyHQlf31Z/OYqQGBZajgledGRW/xxhwiAQO8X/Zk327nqgw== X-Google-Smtp-Source: ABdhPJwMIXxnDn1X5Xfr1fyMK9y7bq/fg97su3JSVrAjsYDF1AdX/54pYTnZ9hwSwhaJ9rEp1sfvdZ74JiJiL1N5iEU= X-Received: by 2002:a5d:67c8:: with SMTP id n8mr12685607wrw.351.1614655336025; Mon, 01 Mar 2021 19:22:16 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Priyanshu Mittal Date: Tue, 2 Mar 2021 08:52:05 +0530 Message-ID: Subject: Re: How to connect arrow flights to spark clusters To: user@arrow.apache.org, emkornfield@gmail.com Content-Type: multipart/alternative; boundary="000000000000bd11bf05bc853db9" --000000000000bd11bf05bc853db9 Content-Type: text/plain; charset="UTF-8" Hi Micah, I tried to understand this repo https://github.com/rymurr/flight-spark-source but I'm not able to build and run this code. Readme file also doesn't explain how to run this code.. So, can anyone help me in running this code? And also is there any documentation related to it? Further, I found very less resources for usage of Arrow flight and like how we can use it in our project. So If there are any implementations of Arrow flight or Spark Flight connector, It would be great if you can share that with us. Thanks, Priyanshu On Sun, Feb 28, 2021 at 10:40 PM Priyanshu Mittal wrote: > Hi, > Thanks for sharing that. I'm looking into it. > > On Sat, 27 Feb, 2021, 10:21 am Micah Kornfield, > wrote: > >> Ryan Murray prototyped something like this a while ago [1]. I'm not sure >> what the current state of this is. >> >> [1] https://github.com/rymurr/flight-spark-source >> >> On Thu, Feb 18, 2021 at 10:15 PM Priyanshu Mittal >> wrote: >> >>> Hi, >>> I want to connect my arrow flight with apache spark clusters so that I >>> can transfer my data between spark clusters and arrow flights. Basically, I >>> have to transfer my arrow data to spark clusters using arrow flights. So, >>> How can I do that? >>> Any help or resources would be appreciated. >>> >>> Thanks, >>> Priyanshu >>> >> --000000000000bd11bf05bc853db9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Micah,

I tried to understand this re= po=C2=A0https://github.com/rymurr/flight-spark-source=C2=A0but I'm= not able to build and run this code. Readme file also doesn't explain = how to run this code.. So, can anyone help me in running this code? And als= o is there any documentation related to it?

Furthe= r, I found very less resources for usage of Arrow flight and like how we ca= n use it in our project. So If there are any implementations of Arrow fligh= t or Spark Flight connector, It would be great if you can share that with u= s.

Thanks,
Priyanshu

On Sun, Feb 28= , 2021 at 10:40 PM Priyanshu Mittal <prmittal.nch@gmail.com> wrote:
Hi,
Thanks = for sharing that. I'm looking into it.=C2=A0

On Sat, 27 Feb, 2021= , 10:21 am Micah Kornfield, <emkornfield@gmail.com> wrote:
=
Ry= an Murray prototyped something like this a while ago [1].=C2=A0 I'm not= sure what the current state of this is.


On Thu, Feb 18, 2021 at 10:15 PM Priyanshu Mittal <p= rmittal.nch@gmail.com> wrote:
Hi,
I want to conne= ct my arrow flight with apache spark clusters so that I can transfer my dat= a between spark clusters and arrow flights. Basically, I have to transfer m= y arrow data to spark clusters using arrow flights. So, How can I do that?<= /div>
Any help or resources would be appreciated.

Thanks,
P= riyanshu
--000000000000bd11bf05bc853db9--