storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toy <noppani...@gmail.com>
Subject Re: Can I emit a Map?
Date Tue, 21 Nov 2017 21:20:52 GMT
We also emit a Map and so far it's been fine. You can also encode the
string if it gets too big but I would measure the performance first
On Tue, Nov 21, 2017 at 13:12 Mauro Giusti <maurgi@microsoft.com> wrote:

> We also emit a map, encoded in Json –
>
> We don’t have any problem with that, but it is only a 28 records map and
> each entry has 5 records with 1 number, so it is kind of small.
>
>
>
> If your map is very large, then you might want to consider transforming
> that into records and emit them for a next level bolt?
>
>
>
> Mauro.
>
>
>
> *From:* Hannum, Daniel [mailto:Daniel_Hannum@PremierInc.com]
> *Sent:* Tuesday, November 21, 2017 12:48 PM
> *To:* user@storm.apache.org
>
>
> *Subject:* Re: Can I emit a Map?
>
>
>
> I forget where I read it, but I was told that there is no de jure size
> limit on tuples. My topology will have tuples 1M in size, and outliers much
> larger than that. I haven’t seen any issues. Obviously at some point, you
> run out of memory…. ☺
>
>
>
> *From: *Marco Costantini <mcsilvio@gmail.com>
> *Reply-To: *"user@storm.apache.org" <user@storm.apache.org>
> *Date: *Tuesday, November 21, 2017 at 3:39 PM
> *To: *"user@storm.apache.org" <user@storm.apache.org>
> *Subject: *Re: Can I emit a Map?
>
>
>
> ****This email did not originate from the Premier, Inc. network. Use
> caution when opening attachments or clicking on URLs.*****
>
>
> .
>
> Thanks. What if it's big? My Strings end up being a couple thousand
> characters long, and each map has around 500 of them. Seems daunting to
> emit such a tuple. Any thoughts on that?
>
>
>
> On Tue, Nov 21, 2017 at 7:23 PM, Stig Rohde Døssing <srdo@apache.org>
> wrote:
>
> I think that as long as you ensure that everything in the map can be
> serialized, it's fine.
>
>
>
> 2017-11-21 18:31 GMT+01:00 Marco Costantini <mcsilvio@gmail.com>:
>
> I have a batching bolt which ultimately makes a Map<String, Set<String>>.
> At a time I decide, I need to emit that map to the next bolt in the
> topology. I've tried it and it works, but I'm curious as to whether this is
> an anti-pattern or if there are problems with doing this that I am unaware
> of.
>
>
>
> Please and thanks,
>
> Marco.
>
>
>
>
>

Mime
View raw message