ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Mashenkov <andrey.mashen...@gmail.com>
Subject Re: Input data is no significant change in multi-threading
Date Tue, 25 Apr 2017 09:46:11 GMT
Hi Woo,

It may be reasonable, if you see, nodes resources utilization is too low
and rising per-node-buffer size have no effect (that means you prepare data
for nodes too slow).
Of course, you should check first if network isn't a bottleneck.

On Tue, Apr 25, 2017 at 10:08 AM, woo charles <ignite.charlesw@gmail.com>
wrote:

> Same data  set mean that I separate original data into 2 parts & input
> them from 2 separate programs.
> E.g. a data set with id  1 - 100. Program A input id 1-50. Program B input
> 51 - 100.
>
> 2017-04-21 17:24 GMT+08:00 Andrey Mashenkov <andrey.mashenkov@gmail.com>:
>
>> Hi Woo,
>>
>> DataStreamer is designed to fill cache with maximum throughput. By
>> default, streamer will not rewrite cache data, until allowOverwite option
>> is set.
>>
>> Why you need to input same set of data? Why do you expected data input
>> time will change significantly with 2 programs compared to 1 if data set is
>> put twice?
>> Or I missed smth?
>>
>> Anyway, if you do not get a speed up but you sure you should, then a
>> bottleneck have to be found at first.
>>
>> On Fri, Apr 21, 2017 at 5:02 AM, woo charles <ignite.charlesw@gmail.com>
>> wrote:
>>
>>> If the data is buffered in client side, the bottleneck should be also in
>>> client side.
>>> If I use 2 programs to input same set data, it should be a significant
>>> change in data input time.
>>> Is it right?
>>>
>>> 2017-04-21 6:46 GMT+08:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
>>>
>>>>
>>>> On Wed, Apr 19, 2017 at 10:16 PM, woo charles <
>>>> ignite.charlesw@gmail.com> wrote:
>>>>
>>>>> When I call addData() in streamer. this data will send & buffer in
>>>>> server node. is that correct?
>>>>> If I correct, this data will buffer in random server node or only the
>>>>> one it directly connected?
>>>>>
>>>>
>>>> addData() will buffer the data on the client side. As a matter of fact,
>>>> there are multiple buffers on the client side, which each buffer associated
>>>> with some server node.
>>>>
>>>> Ignite will never send the data to a random node. The data is always
>>>> sent exactly to the node where it will be cached.
>>>>
>>>> D.
>>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>


-- 
Best regards,
Andrey V. Mashenkov

Mime
View raw message