storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Mahadevan <ar...@apache.org>
Subject Re: Can I emit a Map?
Date Wed, 22 Nov 2017 04:28:28 GMT
I think it would be fine as long as you ensure your map is Immutable or at-least its not changed
after its constructed. I don’t think there would be any issues with sizes.

From:  Toy <noppanit.c@gmail.com>
Reply-To:  "user@storm.apache.org" <user@storm.apache.org>
Date:  Wednesday, November 22, 2017 at 2:50 AM
To:  "user@storm.apache.org" <user@storm.apache.org>
Subject:  Re: Can I emit a Map?

We also emit a Map and so far it's been fine. You can also encode the string if it gets too
big but I would measure the performance first
On Tue, Nov 21, 2017 at 13:12 Mauro Giusti <maurgi@microsoft.com> wrote:
We also emit a map, encoded in Json –

We don’t have any problem with that, but it is only a 28 records map and each entry has
5 records with 1 number, so it is kind of small.

 

If your map is very large, then you might want to consider transforming that into records
and emit them for a next level bolt?

 

Mauro.

 

From: Hannum, Daniel [mailto:Daniel_Hannum@PremierInc.com] 
Sent: Tuesday, November 21, 2017 12:48 PM
To: user@storm.apache.org


Subject: Re: Can I emit a Map?

 

I forget where I read it, but I was told that there is no de jure size limit on tuples. My
topology will have tuples 1M in size, and outliers much larger than that. I haven’t seen
any issues. Obviously at some point, you run out of memory…. ☺

 

From: Marco Costantini <mcsilvio@gmail.com>
Reply-To: "user@storm.apache.org" <user@storm.apache.org>
Date: Tuesday, November 21, 2017 at 3:39 PM
To: "user@storm.apache.org" <user@storm.apache.org>
Subject: Re: Can I emit a Map?

 

****This email did not originate from the Premier, Inc. network. Use caution when opening
attachments or clicking on URLs.*****


.

Thanks. What if it's big? My Strings end up being a couple thousand characters long, and each
map has around 500 of them. Seems daunting to emit such a tuple. Any thoughts on that?

 

On Tue, Nov 21, 2017 at 7:23 PM, Stig Rohde Døssing <srdo@apache.org> wrote:

I think that as long as you ensure that everything in the map can be serialized, it's fine.

 

2017-11-21 18:31 GMT+01:00 Marco Costantini <mcsilvio@gmail.com>:

I have a batching bolt which ultimately makes a Map<String, Set<String>>. At a
time I decide, I need to emit that map to the next bolt in the topology. I've tried it and
it works, but I'm curious as to whether this is an anti-pattern or if there are problems with
doing this that I am unaware of. 

 

Please and thanks,

Marco.

 

 


Mime
View raw message