samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Angelica Garcia-Gutierrez <agar...@transcendinsights.com>
Subject Re: Samza + MongoDb Sharding
Date Fri, 07 Jul 2017 17:25:22 GMT
That is correct, I am augmenting each message in the input stream with
further information from the MongoDB.



On 7/6/17, 5:43 PM, "Jagadish Venkatraman" <jagadish1989@gmail.com> wrote:

>What is your input source you are reading the stream from?  IIUC, you are
>trying to make a remote call to mongo to decorate/ augment each message?
>
>Best,
>Jagdish
>
>On Fri, Jul 7, 2017 at 12:51 AM Angelica Garcia-Gutierrez <
>agarcia@transcendinsights.com> wrote:
>
>> Hello,
>>
>> I have a Samza job that currently makes remote calls to a MongoDB to get
>> additional information about the input stream. For scalability, MongoDB
>>was
>> initially partitioned into 4 shards (more shards will be added as
>>needed).
>> The questions are:
>>
>>   *   Does it make sense to attempt to partition the input stream into
>> multiple partitions such that a given task can consume it and expand the
>> message with information retrieved from a specific MongoDB shard?
>>
>> Can someone please shed some lights?
>>
>> Thanks,
>> Angelica.
>>
>>
>> The information transmitted is intended only for the person or entity to
>> which it is addressed
>> and may contain CONFIDENTIAL material.  If you receive this
>> material/information in error,
>> please contact the sender and delete or destroy the
>>material/information.
>
>-- 
>Sent from my iphone.

The information transmitted is intended only for the person or entity to which it is addressed
and may contain CONFIDENTIAL material.  If you receive this material/information in error,
please contact the sender and delete or destroy the material/information.

Mime
View raw message