ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From woo charles <ignite.charl...@gmail.com>
Subject Re: Input data is no significant change in multi-threading
Date Thu, 20 Apr 2017 05:16:47 GMT
When I call addData() in streamer. this data will send & buffer in server
node. is that correct?
If I correct, this data will buffer in random server node or only the one
it directly connected?

2017-04-19 18:33 GMT+08:00 Andrey Mashenkov <andrey.mashenkov@gmail.com>:

> It may have effect if you prepare data for streamer (call addData) slowly
> and it is possible to utilize more resources for it. Of course remote nodes
> should be able to bear pressure of data.
> Performance can increased, but usually slightly as network will be a
> bottleneck.
>
>
> On Wed, Apr 19, 2017 at 12:29 PM, woo charles <ignite.charlesw@gmail.com>
> wrote:
>
>> Is that mean the performance of input data will not affect if I use 2 IgniteDataStreamer(2
>> client program) to input data as they use the same queue in remote nodes?
>>
>> 2017-04-19 10:02 GMT+08:00 Andrey Mashenkov <andrey.mashenkov@gmail.com>:
>>
>>> Hi Woo,
>>>
>>> IgniteDataStreamer uses per node buffer to make bulk cache updates that
>>> shows much better throughput than single updates.
>>> Also, IgniteDataStreamer send jobs to remote nodes, to utilize multiple
>>> threads on remote nodes.
>>>
>>> In multi-node grid IgniteDataStreamer usually shows better results than
>>> single updates in from multiple threads.
>>>
>>>
>>>
>>> On Wed, Apr 19, 2017 at 4:30 AM, woo charles <ignite.charlesw@gmail.com>
>>> wrote:
>>>
>>>> When I try to input data(80 table, each 10000 records) to a cluster
>>>> with 3 server node(each 2 gb), it only has a small change in time if multi
>>>> thread is performed
>>>> (ie. max decrease from 8s to 6.5s if using IgniteCache)
>>>>
>>>> Is it normal?
>>>>
>>>> Also, I found that multi thread do not affect the data input speed in
>>>> IgniteDataStreamer.
>>>>
>>>> Is it true?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>

Mime
View raw message