ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anil <anilk...@gmail.com>
Subject Re: Data Streamer
Date Sun, 23 Oct 2016 06:12:08 GMT
Thank you Manu for pointing out the issue.

i have created data streamer object for each IgniteCallable task and it
looks good.

one datastreamer could be not be shared with all taks as it is not
serialized.

Thanks

On 22 October 2016 at 22:34, Manu <maxnu00@hotmail.com> wrote:

> Hi,
>
> Your are creating new data streamer on each loop call...
> [...]
>                                 for (int i = 0; i < 1000000; i++){
> //                                      CacheManager.
> getInstallBaseCache().put(name+"-"+i, new
> TestPojo());
>
> CacheManager.getInstance().dataStreamer(CACHE).addData(name+"-"+i, new
> TestPojo());
>                                 }
> [...]
>
> Ignite does this when you call dataStreamer(cache name) method...
> [...]
>   /**
>      * @param cacheName Cache name ({@code null} for default cache).
>      * @return Data loader.
>      */
>     public DataStreamerImpl<K, V> dataStreamer(@Nullable String cacheName)
> {
>         if (!busyLock.enterBusy())
>             throw new IllegalStateException("Failed to create data streamer
> (grid is stopping).");
>
>         try {
>             final DataStreamerImpl<K, V> ldr = new DataStreamerImpl<>(ctx,
> cacheName, flushQ);
>
>             ldrs.add(ldr);
> [...]
>
> So try create a data streamer instance only once.
> [...]
> *IgniteStream stream = CacheManager.getInstance().dataStreamer(CACHE);*
>
>                                 for (int i = 0; i < 1000000; i++){
>                                         stream.addData(name+"-"+i, new
> TestPojo());
>                                 }
> [...]
>
> Another improvement is send data on a "buffered fashion", so you reduce
> calls to cluster... try stream.addData(data); // where data buffer =
> Map<Key,Value>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Data-Streamer-tp8409p8427.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Mime
View raw message