ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Tupitsyn <ptupit...@apache.org>
Subject Re: Effective size limit for cache items in Ignite
Date Mon, 13 Feb 2017 13:15:42 GMT
Hi Raymond,

CopyOnRead setting has no effect in .NET code.
It exists for cases when there are both Java and .NET nodes in a cluster.

In Ignite.NET each cache.Get call causes two things:
1) Copy a piece of memory from Java (serialized data)
2) Deserialize the data into user type instance.

You can avoid deserialization by using Binary Mode:
https://apacheignite-net.readme.io/docs/binary-mode

Pavel

On Mon, Feb 13, 2017 at 6:38 AM, Raymond Wilson <raymond_wilson@trimble.com>
wrote:

> I found this code snippet in the Ignite v1.8 source:
>
>
>
> This appears to be the core place where the value of CopyOnRead has an
> effect, though I don’t understand the context of the other logical
> conditions around it; it seems that CopyOnRead is dependent on other
> configuration state before it will be honoured. I’m still digging to locate
> the detailed help around this setting to understand these other conditions.
>
>     /** {@inheritDoc} */
>
>     @Override public CacheObjectContext contextForCache(CacheConfiguration
> ccfg) throws IgniteCheckedException {
>
>         assert ccfg != null;
>
>
>
>         CacheMemoryMode memMode = ccfg.getMemoryMode();
>
>
>
>         boolean storeVal = !ccfg.isCopyOnRead() || (!isBinaryEnabled(ccfg)
> &&
>
>             (GridQueryProcessor.isEnabled(ccfg) || ctx.config().
> isPeerClassLoadingEnabled()));
>
>
>
>         CacheObjectContext res = new CacheObjectContext(ctx,
>
>             ccfg.getName(),
>
>             ccfg.getAffinityMapper() != null ? ccfg.getAffinityMapper() :
> new GridCacheDefaultAffinityKeyMapper(),
>
>             ccfg.isCopyOnRead() && memMode != OFFHEAP_VALUES,
>
>             storeVal,
>
>             ctx.config().isPeerClassLoadingEnabled() &&
> !isBinaryEnabled(ccfg));
>
>
>
>         ctx.resource().injectGeneric(res.defaultAffMapper());
>
>
>
>         return res;
>
>     }
>
>
>
> Thanks,
>
> Raymond.
>
>
>
> *From:* Raymond Wilson [mailto:raymond_wilson@trimble.com]
> *Sent:* Monday, February 13, 2017 2:43 PM
> *To:* user@ignite.apache.org
> *Subject:* RE: Effective size limit for cache items in Ignite
>
>
>
> Ah, I found the CopyOnRead flag in the cache configuration.
>
>
>
> Unfortunately, it seems to have the same behaviour regardless of the
> setting for this flag.
>
>
>
> If I create an example like the below, it seems that querying the same
> element from the cache many times takes about the same amount of time in
> both cases. Visual Studio also reports large numbers of GC episodes while
> it cleans up the large freed MyCacheClass instances. Is this flag only
> applicable to Java contexts? I did also try setting KeepBinaryInStore to
> true, though there was no noticeable difference.
>
>
>
> [Serializable]
>
>     public class MyCacheClass
>
>     {
>
>         public String name = String.Empty;
>
>         private byte[] localData = null;
>
>
>
>         public MyCacheClass(String _name)
>
>         {
>
>             name = _name;
>
>             localData = new byte[4000000];
>
>         }
>
>     }
>
>
>
>     class Program
>
>     {
>
>         static void Main(string[] args)
>
>         {
>
>             IIgnite ignite = Ignition.Start();
>
>
>
>             // Add a cache to Ignite
>
>             ICache<String, MyCacheClass> cache = ignite.CreateCache<String,
> MyCacheClass>
>
>                 (new CacheConfiguration()
>
>                 {
>
>                     Name = "TestCache",
>
>                     CopyOnRead = false,
>
>                     KeepBinaryInStore = true
>
>                 });
>
>
>
>             // Add a cache item
>
>             cache.Put("First", new MyCacheClass("FirstItem"));
>
>
>
>             // query back the cache items
>
>             for (int i = 0; i < 30000; i++)
>
>             {
>
>                 MyCacheClass first = cache.Get("First");
>
>             }
>
>       }
>
>
>
>
>
> *From:* Raymond Wilson [mailto:raymond_wilson@trimble.com
> <raymond_wilson@trimble.com>]
> *Sent:* Monday, February 13, 2017 11:35 AM
> *To:* user@ignite.apache.org
> *Subject:* Effective size limit for cache items in Ignite
>
>
>
> Hi,
>
>
>
> What is the practical size limit for items in an Ignite cache?
>
>
>
> I suspect the answer is something “As large as the memory you have to hold
> it”, but my question is more aimed at the practicality of large items in a
> cache due to the overhead of pulling copies of the items out of the cache
> in response to a Cache.Get() request.
>
>
>
> For instance, let’s say I had cache items in the 64Kb size range, and had
> requests that commonly refer to those cache items to perform some work on
> them in response to a request. Will each Cache.Get() request require an
> extraction and repackaging of the cache item prior to handing it back to
> the caller as a new (copied) version of that cache item, or is there a way
> for just a reference to the cache item to be returned to the caller?
>
>
>
> I understand there is a way to designate the information in a cache as
> just blobs of data with no serialisation semantics. In this case does a
> Cache.Get() return a pointer or a copy (with a local locking semantic to
> prevent change)?
>
>
>
> Thanks,
>
> Raymond.
>
>
>

Mime
View raw message