ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy Pavlov (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (IGNITE-6699) Optimize client-side data streamer performance
Date Tue, 26 Jun 2018 17:51:24 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-6699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Dmitriy Pavlov updated IGNITE-6699:
    Fix Version/s:     (was: 2.6)

> Optimize client-side data streamer performance
> ----------------------------------------------
>                 Key: IGNITE-6699
>                 URL: https://issues.apache.org/jira/browse/IGNITE-6699
>             Project: Ignite
>          Issue Type: Task
>          Components: streaming
>    Affects Versions: 2.3
>            Reporter: Vladimir Ozerov
>            Assignee: Ivan Fedotov
>            Priority: Major
>              Labels: performance
>             Fix For: 2.7
> Currently if a user has several server nodes and a single client node with single thread
pushing data to streamer, he will not be able to load data at maximum speed. On the other
hand, if he start several data loading threads, throughput will increase. 
> One of root causes of this is bad data streamer design. Method {{IgniteDataStreamer.addData(K,
V)}} returns new feature for every operation, this is too fine grained approach. Also it generates
a lot of garbage and causes contention on streamer internals. 
> Proposed implementation flow:
> 1) Compare performance of {{addData(K, V)}} vs {{addData(Collection)}} methods from one
thread in distributed environment. The latter should show considerably higher throughput.
> 2) Users should receive per-batch features, rather than per-key. 
> 3) Try caching thread data in some collection until it is large enough to avoid contention
and unnecessary allocations.

This message was sent by Atlassian JIRA

View raw message