From user-return-90-archive-asf-public=cust-asf.ponee.io@arrow.apache.org Mon Feb 11 20:33:13 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 697C9180648 for ; Mon, 11 Feb 2019 21:33:13 +0100 (CET) Received: (qmail 98169 invoked by uid 500); 11 Feb 2019 20:33:12 -0000 Mailing-List: contact user-help@arrow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@arrow.apache.org Delivered-To: mailing list user@arrow.apache.org Received: (qmail 98159 invoked by uid 99); 11 Feb 2019 20:33:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Feb 2019 20:33:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 208391807E0 for ; Mon, 11 Feb 2019 20:33:12 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.801 X-Spam-Level: * X-Spam-Status: No, score=1.801 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 5YG-VUggXHx3 for ; Mon, 11 Feb 2019 20:33:09 +0000 (UTC) Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id ADD1F60E01 for ; Mon, 11 Feb 2019 20:33:09 +0000 (UTC) Received: by mail-ot1-f45.google.com with SMTP id z19so496394otm.2 for ; Mon, 11 Feb 2019 12:33:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=y+gAfbafIqJE+mDduFPmIfjqrmqcwGObaJtPn/h5LM0=; b=B/IfeMjBp05AJShtHsHQOP1wQoIpwtvna2HTE/E4kKd9WEOdmuL6p+gO+aNfByYoCC GiXwGbyvwCCPRfhr0r5EDyBAw6Ssts+P8xir5KdJk6zU7MT4fyrPJSUwQzSzsmjrjucA Hvu8N/P7JNX/0pSueAAZ0kPOzr/Rr9dANEozlDHtYVgFk1UXRNEILFSLCBQLlP7t2r0S wRuQD/EM9263vBkHGLj2GuUR1IgontTbBtUx8N/IYQ4ha5+/UvRB9gkzloMVbmp3zbpu GJnwa8x4z4fWuwKkeN4i3RQ9rKr3P5xHcOtVzyVT2IMPOE+4Oar5K5kBePi6Je2lIOvF AZ5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=y+gAfbafIqJE+mDduFPmIfjqrmqcwGObaJtPn/h5LM0=; b=U655FfiHwR/Vuo+CALRTPEWcbPfR2dCFf/3wSlG9eeEbUl7R7NEUfpT8aJYY8vrlU2 lCyJHPmQBQvThGV8g5uxMiGNZG2ef2P3g/a3jidD2nd++Fj4wt6VAJV9WJBTkzs5KDCU 35iLZMI7KayrmAMC53aopQY81a6ZShJbN69YUsF8iEYFRkz1H42gf/yXRASd4eHzhfbL J5RD/qu9gr8GFi102+g42KMPe2r90qiZW3WPeiQEyRIiVkySqLmkIDJ3SifOgY4VZyG2 9iAxBbX48QyxCKQjYuiuwZMEg63lQB4QEaBelTYCvL742j3rPgfVzAs4XHVd2jFEL2Z/ up7Q== X-Gm-Message-State: AHQUAuYZUEzl0KXGRbx8K/TuAUG6hPtjkAbkzI96ClJeCxC1gwNOwwDw ZEOkmqrYnoBLWJYSQZo+jAetnXYUhHSktg1bcTVHCldj X-Google-Smtp-Source: AHgI3IYiNd5b4jNePtAh/mriB79N9uHwOT6/Ybj8ge6ZoiyE7XmIJFqbRbKE5FC450cv0sOJPKTTHsoltlykY5drTQY= X-Received: by 2002:a9d:760f:: with SMTP id k15mr49033otl.149.1549917188497; Mon, 11 Feb 2019 12:33:08 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Robert Nishihara Date: Mon, 11 Feb 2019 12:32:56 -0800 Message-ID: Subject: Re: Reading Tensor from Plasma in Java To: user@arrow.apache.org Content-Type: multipart/alternative; boundary="00000000000073a4b50581a43770" --00000000000073a4b50581a43770 Content-Type: text/plain; charset="UTF-8" I'd suggest taking a look at how the custom TF op reads from the plasma store https://github.com/apache/arrow/blob/d7b6ca7feb8f6419cee7069dcef20252579ea107/python/pyarrow/tensorflow/plasma_op.cc#L238 . You can directly read the object using the Java Plasma client (and then create a TF Tensor from the resulting buffer). It may also be possible to use the custom TF op in Java though I'm not sure what's involved there. On Sun, Feb 10, 2019 at 6:43 AM Clive Cox wrote: > > I want to read an Arrow Tensor saved from python to Plasma into Java. > I thought it might be directly saved as a Tensor Flatbuffer but it looks > like not. Can someone say how this can be done or the FlatBuffers type to > use on the returned bytes from Plasma? > > Thanks, > > Clive > --00000000000073a4b50581a43770 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I'd suggest taking a look at how the = custom TF op reads from the plasma store https://github.com/apache/arrow/blob/d7b6ca7feb8f6= 419cee7069dcef20252579ea107/python/pyarrow/tensorflow/plasma_op.cc#L238= .

You can directly read the object usi= ng the Java Plasma client (and then create a TF Tensor from the resulting b= uffer). It may also be possible to use the custom TF op in Java though I= 9;m not sure what's involved there.

On Sun, Feb 10, 2019 at 6:= 43 AM Clive Cox <cc@seldon.io> wr= ote:

I want to read an Arrow Tensor save= d from python to Plasma into Java.
I thought it might be directly= saved as a Tensor Flatbuffer but it looks like not. Can someone say how th= is can be done or the FlatBuffers type to use on the returned bytes from Pl= asma?

Thanks,

Clive
--00000000000073a4b50581a43770--