Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id BCB4F200D43 for ; Tue, 21 Nov 2017 22:21:14 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id BB306160BFC; Tue, 21 Nov 2017 21:21:14 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D99BC160BE3 for ; Tue, 21 Nov 2017 22:21:13 +0100 (CET) Received: (qmail 22204 invoked by uid 500); 21 Nov 2017 21:21:07 -0000 Mailing-List: contact user-help@storm.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.apache.org Delivered-To: mailing list user@storm.apache.org Received: (qmail 22193 invoked by uid 99); 21 Nov 2017 21:21:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 21 Nov 2017 21:21:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id EB50C18071E for ; Tue, 21 Nov 2017 21:21:06 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.401 X-Spam-Level: X-Spam-Status: No, score=-0.401 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id FfU77D5C1m0X for ; Tue, 21 Nov 2017 21:21:05 +0000 (UTC) Received: from mail-qk0-f171.google.com (mail-qk0-f171.google.com [209.85.220.171]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 83A775FAC9 for ; Tue, 21 Nov 2017 21:21:04 +0000 (UTC) Received: by mail-qk0-f171.google.com with SMTP id 136so14049106qkd.4 for ; Tue, 21 Nov 2017 13:21:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=4DIuT4k0Xr2BYYrndYRbeV4MrDldeC9dQAZYhrnqIAc=; b=LypYDG7QfGhETU9ypsCKNMSYp247Du7uP6uhweAh5UTnLeCW6iMZd4qoWUtMeHVei7 55B7E9hiGoHYC6xtzmagixLzW2W0eYAjVq6USG8eTqikDyzQxQdknrwOZuiQQqcqyyUC Zti7q/Tp1R1y/Z3s/tzQsMWuaVMaaKiFkIuHBh08oKa/7ZWmBsmBguCnKBKqJs4gwg59 Zhn7ZLjAkhGDo8iE+WQZ37YpyVdbhFs8CFZzxJEshJVIoyHBHsqT3HUgIjZQ+AMqIrbf NQ+VfKmgdV4C0EC3YFjVHGA9tn1phRXJhcdGDj1u9jCOsE8jT5K4Z7OXoAO6uTh7Rn+P jKJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=4DIuT4k0Xr2BYYrndYRbeV4MrDldeC9dQAZYhrnqIAc=; b=Cb3UOzFZvvXc8PEGJ9ZQ1iZWHaZfyRCm6Fckbx/HRkATKg4+0Vb1fTjelRVGrkJXtf Tf2Y3//lcdJKyT1/Sd+yIiKPdBrpoNgnpQ+ztsGUCPUHwJ667Lnc7CiT3PEkcZ0Ttdau yD6o5HFvijZQgKmWiFVBqdY6xlJKVDl1Cpuw3ZAvQmtaBybbOOsxDiaC4Fq0c3SlQljg DK8bIH1YGd5R0EAy56farNu5Obm1DPug3JswokDvzFYEJZ+u9RMg2yNr5r10cZRoa+FE 696X46244OihidyILMXjWde514wFIelEqxXDiZ5N8hM3hNjLXqARffjrm+mNa91uSzRo JOQg== X-Gm-Message-State: AJaThX567Z2uN6aEOKGqxTr6YJIou4xvvqvBjYJDZ+8LiDzGhNFYi7hf YLkEiBAngbVAQptXaPYGVUVOVV04ELAKFsTs5qEDPg== X-Google-Smtp-Source: AGs4zMZw/mJlc7CWzoK37+ldLUsHCUdSaU8akd0bCGWyaulkGYJuhImsM6eym+3IZIilvd6Xzq2rtS/7Hij2OeGVVG8= X-Received: by 10.55.23.195 with SMTP id 64mr70016qkx.262.1511299263225; Tue, 21 Nov 2017 13:21:03 -0800 (PST) MIME-Version: 1.0 References: <87DC2644-0EF4-4AC5-AA78-556DA42CE4C4@premierinc.com> In-Reply-To: From: Toy Date: Tue, 21 Nov 2017 21:20:52 +0000 Message-ID: Subject: Re: Can I emit a Map? To: user@storm.apache.org Content-Type: multipart/alternative; boundary="001a1147f1a2bbe45e055e84c7ed" archived-at: Tue, 21 Nov 2017 21:21:14 -0000 --001a1147f1a2bbe45e055e84c7ed Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable We also emit a Map and so far it's been fine. You can also encode the string if it gets too big but I would measure the performance first On Tue, Nov 21, 2017 at 13:12 Mauro Giusti wrote: > We also emit a map, encoded in Json =E2=80=93 > > We don=E2=80=99t have any problem with that, but it is only a 28 records = map and > each entry has 5 records with 1 number, so it is kind of small. > > > > If your map is very large, then you might want to consider transforming > that into records and emit them for a next level bolt? > > > > Mauro. > > > > *From:* Hannum, Daniel [mailto:Daniel_Hannum@PremierInc.com] > *Sent:* Tuesday, November 21, 2017 12:48 PM > *To:* user@storm.apache.org > > > *Subject:* Re: Can I emit a Map? > > > > I forget where I read it, but I was told that there is no de jure size > limit on tuples. My topology will have tuples 1M in size, and outliers mu= ch > larger than that. I haven=E2=80=99t seen any issues. Obviously at some po= int, you > run out of memory=E2=80=A6. =E2=98=BA > > > > *From: *Marco Costantini > *Reply-To: *"user@storm.apache.org" > *Date: *Tuesday, November 21, 2017 at 3:39 PM > *To: *"user@storm.apache.org" > *Subject: *Re: Can I emit a Map? > > > > ****This email did not originate from the Premier, Inc. network. Use > caution when opening attachments or clicking on URLs.***** > > > . > > Thanks. What if it's big? My Strings end up being a couple thousand > characters long, and each map has around 500 of them. Seems daunting to > emit such a tuple. Any thoughts on that? > > > > On Tue, Nov 21, 2017 at 7:23 PM, Stig Rohde D=C3=B8ssing > wrote: > > I think that as long as you ensure that everything in the map can be > serialized, it's fine. > > > > 2017-11-21 18:31 GMT+01:00 Marco Costantini : > > I have a batching bolt which ultimately makes a Map>. > At a time I decide, I need to emit that map to the next bolt in the > topology. I've tried it and it works, but I'm curious as to whether this = is > an anti-pattern or if there are problems with doing this that I am unawar= e > of. > > > > Please and thanks, > > Marco. > > > > > --001a1147f1a2bbe45e055e84c7ed Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable We also emit a Map and so far it's been fine. You can also encode the s= tring if it gets too big but I would measure the performance first
On Tue, Nov 21, 2017 at 13:12 Mauro = Giusti <maurgi@microsoft.com= > wrote:

We also emit a map, encoded in Json =E2=80=93=

We don=E2=80=99t have any problem with that, but it = is only a 28 records map and each entry has 5 records with 1 number, so it = is kind of small.

=C2=A0

If your map is very large, then you might want to co= nsider transforming that into records and emit them for a next level bolt?<= u>

=C2=A0

Mauro.

=C2=A0

From: Hannum, Daniel [mailto:Daniel_Hannum@PremierInc.= com]
Sent: Tuesday, November 21, 2017 12:48 PM
To: user@= storm.apache.org


Subject: Re: Can I emit a Map?

=C2=A0

I forget where I read it, but I was told that there = is no de jure size limit on tuples. My topology will have tuples 1M in size= , and outliers much larger than that. I haven=E2=80=99t seen any issues. Ob= viously at some point, you run out of memory=E2=80=A6. =E2=98=BA=

=C2=A0

From= : Marco Costantini = <mcsilvio@gmail.= com>
Reply-To: "user@storm.apache.org" <user@storm.apache.org>
Date: Tuesday, November 21, 2017 at 3:39 PM
To: "user@storm.apache.org" <user@storm.apache.org>
Subject: Re: Can I emit a Map?

=C2=A0

****This email did not originate from the Premier, I= nc. network. Use caution when opening attachments or clicking on URLs.*****=


.

Thanks. What if it's big? My Strings end up bein= g a couple thousand characters long, and each map has around 500 of them. S= eems daunting to emit such a tuple. Any thoughts on that?

=C2=A0

On Tue, Nov 21, 2017 at 7:23 PM, Stig Rohde D=C3=B8s= sing <srdo@apache.o= rg> wrote:

I think that as long as you ensure that everything i= n the map can be serialized, it's fine.

=C2=A0

2017-11-21 18:31 GMT+01:00 Marco Costantini <mcsilvio@gmail.com&= gt;:

I have a batching bolt which ultimately makes a Map&= lt;String, Set<String>>. At a time I decide, I need to emit that m= ap to the next bolt in the topology. I've tried it and it works, but I&= #39;m curious as to whether this is an anti-pattern or if there are problems with doing this that I am unaware of. =

=C2=A0

Please and thanks,

Marco.

=C2=A0

=C2=A0

--001a1147f1a2bbe45e055e84c7ed--