Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 3ADB9200D4F for ; Wed, 22 Nov 2017 05:28:41 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 3969C160C0E; Wed, 22 Nov 2017 04:28:41 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 57703160BFC for ; Wed, 22 Nov 2017 05:28:40 +0100 (CET) Received: (qmail 9778 invoked by uid 500); 22 Nov 2017 04:28:38 -0000 Mailing-List: contact user-help@storm.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@storm.apache.org Delivered-To: mailing list user@storm.apache.org Received: (qmail 9769 invoked by uid 99); 22 Nov 2017 04:28:38 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 22 Nov 2017 04:28:38 +0000 Received: from [192.168.0.4] (unknown [115.99.103.196]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 320651A006D for ; Wed, 22 Nov 2017 04:28:36 +0000 (UTC) User-Agent: Microsoft-MacOutlook/f.16.0.160506 Date: Wed, 22 Nov 2017 09:58:28 +0530 Subject: Re: Can I emit a Map? From: Arun Mahadevan Sender: Arun Iyer To: "user@storm.apache.org" Message-ID: <7963CFF3-5A45-4F08-92A8-BD4C1E9BB3A8@hortonworks.com> Thread-Topic: Can I emit a Map? References: <87DC2644-0EF4-4AC5-AA78-556DA42CE4C4@premierinc.com> In-Reply-To: Mime-version: 1.0 Content-type: multipart/alternative; boundary="B_3594189518_753281857" archived-at: Wed, 22 Nov 2017 04:28:41 -0000 > This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. --B_3594189518_753281857 Content-type: text/plain; charset="UTF-8" Content-transfer-encoding: quoted-printable I think it would be fine as long as you ensure your map is Immutable or at-= least its not changed after its constructed. I don=E2=80=99t think there would be = any issues with sizes. From: Toy Reply-To: "user@storm.apache.org" Date: Wednesday, November 22, 2017 at 2:50 AM To: "user@storm.apache.org" Subject: Re: Can I emit a Map? We also emit a Map and so far it's been fine. You can also encode the strin= g if it gets too big but I would measure the performance first On Tue, Nov 21, 2017 at 13:12 Mauro Giusti wrote: We also emit a map, encoded in Json =E2=80=93 We don=E2=80=99t have any problem with that, but it is only a 28 records map and = each entry has 5 records with 1 number, so it is kind of small. =20 If your map is very large, then you might want to consider transforming tha= t into records and emit them for a next level bolt? =20 Mauro. =20 From: Hannum, Daniel [mailto:Daniel_Hannum@PremierInc.com]=20 Sent: Tuesday, November 21, 2017 12:48 PM To: user@storm.apache.org Subject: Re: Can I emit a Map? =20 I forget where I read it, but I was told that there is no de jure size limi= t on tuples. My topology will have tuples 1M in size, and outliers much larg= er than that. I haven=E2=80=99t seen any issues. Obviously at some point, you run = out of memory=E2=80=A6. =E2=98=BA =20 From: Marco Costantini Reply-To: "user@storm.apache.org" Date: Tuesday, November 21, 2017 at 3:39 PM To: "user@storm.apache.org" Subject: Re: Can I emit a Map? =20 ****This email did not originate from the Premier, Inc. network. Use cautio= n when opening attachments or clicking on URLs.***** . Thanks. What if it's big? My Strings end up being a couple thousand charact= ers long, and each map has around 500 of them. Seems daunting to emit such a= tuple. Any thoughts on that? =20 On Tue, Nov 21, 2017 at 7:23 PM, Stig Rohde D=C3=B8ssing wrot= e: I think that as long as you ensure that everything in the map can be serial= ized, it's fine. =20 2017-11-21 18:31 GMT+01:00 Marco Costantini : I have a batching bolt which ultimately makes a Map>. A= t a time I decide, I need to emit that map to the next bolt in the topology.= I've tried it and it works, but I'm curious as to whether this is an anti-p= attern or if there are problems with doing this that I am unaware of.=20 =20 Please and thanks, Marco. =20 =20 --B_3594189518_753281857 Content-type: text/html; charset="UTF-8" Content-transfer-encoding: quoted-printable
I think it would be= fine as long as you ensure your map is Immutable or at-least its not change= d after its constructed. I don’t think there would be any issues with = sizes.

From: Toy <noppanit.c@gmail.com>
Reply-To: "user@storm.apache.org" <us= er@storm.apache.org>
Date: = Wednesday, November 22, 2017 at 2:50 AM
To= : "user@storm.apache.org"= <user@storm.apache.org>Subject: Re: Can I emit a Map?

We also emit a Map and so far it's been fine. You can also encode the strin= g if it gets too big but I would measure the performance first
On Tue, Nov 21, 2017 at 13:12 Mauro Giusti <= ;maurgi@microsoft.com> wrote:

We also emit a map, encoded in Json –

We don’t have any problem with that, but it is only = a 28 records map and each entry has 5 records with 1 number, so it is kind o= f small.

 

If your map is very large, then you might want to consider t= ransforming that into records and emit them for a next level bolt?=

 

Ma= uro.

 


Subject: Re: Can I emit a Map?

 

I forget where I read it, but I w= as told that there is no de jure size limit on tuples. My topology will have= tuples 1M in size, and outliers much larger than that. I haven’t seen= any issues. Obviously at some point, you run out of memory…. =E2=98=BA<= u>

 

From: = Marco Costantini <mcsilvio@gmail.com>
Rep= ly-To: "user@stor= m.apache.org" <user@storm.apache.org>
Date: Tuesday, November 21, 2017 at= 3:39 PM
To: "user@storm.apache.org" <user@storm.apache.org>
Subject: Re: Can I emit= a Map?

 = ;

****This email did not originate= from the Premier, Inc. network. Use caution when opening attachments or cli= cking on URLs.*****


.

Thanks. What if it's big= ? My Strings end up being a couple thousand characters long, and each map ha= s around 500 of them. Seems daunting to emit such a tuple. Any thoughts on t= hat?

 

On Tue, Nov 21, 2017 at 7:23 PM, Stig Rohde D=C3=B8s= sing <srdo@apache.org> wrote:

&nbs= p;

2017-11-21 18:31 GMT+01:00 Marco Cost= antini <mcsilvio@gmai= l.com>:

I have a= batching bolt which ultimately makes a Map<String, Set<String>>= . At a time I decide, I need to emit that map to the next bolt in the topolo= gy. I've tried it and it works, but I'm curious as to whether this is an ant= i-pattern or if there are problems with doing this that I am unaware of. =

 

Please and thanks,

Marco.

 

 

<= /div> --B_3594189518_753281857--