From dev-return-5693-archive-asf-public=cust-asf.ponee.io@asterixdb.apache.org Wed Apr 11 02:36:51 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1038618064C for ; Wed, 11 Apr 2018 02:36:50 +0200 (CEST) Received: (qmail 74767 invoked by uid 500); 11 Apr 2018 00:36:50 -0000 Mailing-List: contact dev-help@asterixdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.apache.org Delivered-To: mailing list dev@asterixdb.apache.org Received: (qmail 74751 invoked by uid 99); 11 Apr 2018 00:36:49 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 11 Apr 2018 00:36:49 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 1031BC013E for ; Wed, 11 Apr 2018 00:36:49 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 9G5togWx6uwl for ; Wed, 11 Apr 2018 00:36:47 +0000 (UTC) Received: from mail-wm0-f50.google.com (mail-wm0-f50.google.com [74.125.82.50]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 14C745F5FA for ; Wed, 11 Apr 2018 00:36:47 +0000 (UTC) Received: by mail-wm0-f50.google.com with SMTP id g8so683296wmd.2 for ; Tue, 10 Apr 2018 17:36:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=2trH1zjWvqDUnrZ+CLJHIsmhT6YVAKxYetaWFxs89S4=; b=CKsQ2JsT4HTFFr8SiOPkxwQoGSyxTAqWdXDKMoIE+QSnAgwRtkj5C7sRTiU8FzYA5Q WkuC1F2V/wbTwzOLYCUzclmQ2h6ltG9yXUS/rYM4jnDXNJQXZsjIQHhG8oTh1wd9pzCe WsEj0CJXO87zPA9o54JIYh0rfsEvZl1tl7HchrjF/NmL+rwGpQDxlRApd8TiFRAI9LgV R2zBMpTVEgPvLmXEGVagGkGsXpeeafNUI/3hHHHTKF7KqF4JKk5hzVHECYOigY833SM7 A/iq4yjNsk2ATfIOT7uLxvkJBHfyiwzUO1wN7y7CHEqYVolQ/E2BpRa77F16pdzm95In wYzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=2trH1zjWvqDUnrZ+CLJHIsmhT6YVAKxYetaWFxs89S4=; b=r+VFrLOEXRtpzD43zGVJ6jxRxn5q/YDnkkp1YaUGV07FVSmo49CYcKQLURN7f2Gt4j zMsoTim5fEMTe8+vb55otNqja3b0tQ9JdIHPmgQSpxsb2QOPo9HSqgzSjLheMHUmnaz6 L3xuv0agjQ4AffQuEikyHQUwKN11Bj/2Q9ZGt+nwRGthsirx2MBvEQs8GxOlJYmTcLLd 1rz89D6XVolG+RwiD/j73pZpoJzJGdzpCsDS+eV1oS33Vjr0wu+5LcEye6m9mRx22Uh3 soUX7sQmI04Be6r6/7xAcZk1jBc+YAkBlWpJQjutLksZ5MQZgqakbj1PQ494NDtsy7VX SWUg== X-Gm-Message-State: ALQs6tAVgYbUbd8TiUwNVD51mZPDzfetdupFYXLMVft8p9FdVVi1dlGH lAYOIbvX7AaD32xZtopZvERW4XtvvovGusG6jZjCQg== X-Google-Smtp-Source: AIpwx48FONQ3EChHmZfd7zfb7HWffxDcr1pqUS38pvBwd0spS5ZNnpF2hXt6QOYSQU2D9hXjJkIEPcU2olszBI/iDaQ= X-Received: by 10.80.138.138 with SMTP id j10mr6556529edj.36.1523407005910; Tue, 10 Apr 2018 17:36:45 -0700 (PDT) MIME-Version: 1.0 Received: by 10.80.183.137 with HTTP; Tue, 10 Apr 2018 17:36:15 -0700 (PDT) In-Reply-To: References: <819df5b7-61b1-91f2-cacc-04d5e639d888@gmail.com> From: Taewoo Kim Date: Tue, 10 Apr 2018 17:36:15 -0700 Message-ID: Subject: Re: Is there an easier way to wrap/unwrap the entire tuple as a ByteBuffer? To: dev@asterixdb.apache.org Content-Type: multipart/alternative; boundary="94eb2c0ef4246f8b18056987d59f" --94eb2c0ef4246f8b18056987d59f Content-Type: text/plain; charset="UTF-8" Hello Ahmed, This doc might help. https://code.google.com/archive/p/asterixdb/wikis/Serialization.wiki Best, Taewoo On Tue, Apr 10, 2018 at 5:30 PM, Ahmed Eldawy wrote: > Mike, > > What you're suggesting makes more sense. We just don't know how to do it :) > BTW, is there any document that describes the binary format of the > frame/tuple/fields? I was able to find out some information myself by > digging into the code but if there is a document or page that describes > this it can be of a great help. > > On Tue, Apr 10, 2018 at 12:01 PM, Mike Carey wrote: > > > Naive (me as a stupid observer :-)) question: Is there a reason to > > wrap/unwrap instead of extend/unextend? (I.e., couldn't you add an > > additional Hyracks tuple field and then project it away - i.e., expand > and > > contract the tuple horizontally rather than nesting and unnesting it?) > > > > > > > > On 4/10/18 11:10 AM, Chen Luo wrote: > > > >> Hi, > >> > >> You can try IFrameFieldAppender (and its implementation > >> FrameFixedFieldAppender) to directly append wrapped tuple (field by > field) > >> to the output buffer, without going through the array tuple builder. But > >> in > >> general, because of the tuple format, I'm not sure there is a more > >> efficient way to wrap/unwrap tuples directly. > >> > >> Best regards, > >> Chen Luo > >> > >> On Tue, Apr 10, 2018 at 10:33 AM, Muhammad Abu Bakar Siddique < > >> msidd005@ucr.edu> wrote: > >> > >> Hi Dev, > >>> I'm working on a Hyracks application for parallel random sampling which > >>> consists of two operators. The first operator generates and appends a > new > >>> field to each tuple while the second operator processes that additional > >>> field and removes it before writing the final output. So, the output of > >>> the > >>> second operator should have the same format of the input of the first > >>> operator. In other words, I want the first operator to wrap the tuple > >>> as-is > >>> and add an additional field while the second operator should remove and > >>> unwrap the tuple. Currently, I use the FrameTupleAppender and > >>> ArrayTupleAppender where I have to add each field in the input record > >>> separately but it seems to be an overhead in the code. Is there an > easier > >>> way to wrap/unwrap the entire tuple as a ByteBuffer without having to > >>> worry > >>> about the individual fields inside it? > >>> > >>> > > > > > -- > > Ahmed Eldawy > Assistant Professor > http://www.cs.ucr.edu/~eldawy > Tel: +1 (951) 827-5654 > --94eb2c0ef4246f8b18056987d59f--