ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: How to know memory used by a cache or a set
Date Tue, 25 Jun 2019 23:28:16 GMT
Looks like a mess.

Alex Goncharuk, Nikolay Izhikov, considering the latest changes and new
metrics & tracing framework, how would they one get cache/table memory size
via a simple metric? There should be a way to make it workable without
hacks like cachegroups, etc.


-
Denis


On Wed, Jun 19, 2019 at 3:29 AM Alex Plehanov <plehanov.alex@gmail.com>
wrote:

> Denis,
>
> Documentation for memory usage calculation covers another case (memory
> usage by the node).
> There is no ability (AFAIK) in released Ignite versions to calculate
> memory used by a cache or cache group when persistence is disabled.
> Dedicated data region can be used for some of the caches in some cases and
> metrics can be collected for this data region, but when the cache is
> destroyed (or data is deleted) memory is not deallocated, it's going to
> reuse list.
> There is a new metric implemented DataRegionMetrics#getTotalUsedPages
> (count of allocated pages minus count of pages in the reuse list) which
> will partially help to solve Yann's problem, but this metric will be
> available only in the next Ignite release.
> Also, as a temporary workaround, some internal API can be used to get a
> count of pages in the reuse list and calculate total used pages by data
> region manually.
>
> ср, 19 июн. 2019 г. в 07:59, Denis Magda <dmagda@gridgain.com>:
>
>> + dev list
>>
>> Ignite developers,
>>
>> Seems that the present solution for memory calculation doesn't work (check
>> the thread):
>>
>> https://apacheignite.readme.io/v2.5/docs/memory-metrics#section-memory-usage-calculation
>> <https://apacheignite.readme.io/docs/cache-metrics>
>>
>> Was it really broken?
>>
>> --
>> Denis Magda
>>
>>
>> On Thu, Jun 13, 2019 at 9:45 AM yann Blazart <yann.blazart@gmail.com>
>> wrote:
>>
>> > Ok, but I can't create dynamically a data region ? Because each time I
>> > receive a new file to process, I create a cachegroup to handle it, then
>> I
>> > clean everything.
>> >
>> > Le jeu. 13 juin 2019 à 13:28, Alex Plehanov <plehanov.alex@gmail.com>
a
>> > écrit :
>> >
>> >> Hello,
>> >>
>> >> It's a known issue [1]. Now you can get cache group size via JMX only
>> if
>> >> persistence is used.
>> >> If persistence is not used you can get allocated size only for data
>> >> region (but you can have more then one data region and assign cache
>> groups
>> >> to data regions any way you want)
>> >>
>> >> [1] : https://issues.apache.org/jira/browse/IGNITE-8517
>> >>
>> >> ср, 12 июн. 2019 г. в 16:00, yann.blazart@externe.bnpparibas.com
<
>> >> yann.blazart@externe.bnpparibas.com>:
>> >>
>> >>> Hello, I'm back.
>> >>>
>> >>> Well, I need to get memory used by each execution of my process, so
I
>> >>> put all involved caches into the same cacheGroup.
>> >>>
>> >>> If I use the CacheGroupMetricsBean, the size gave to me is 0 !
>> >>> If I enable persistence on DataRegion, I get size, but I don't want
to
>> >>> use the persistence enabled.
>> >>>
>> >>> Is it a bug ?
>> >>>
>> >>> regards.
>> >>>
>> >>> > Le 27 mai 2019 à 11:09, ibelyakov <igor.belyakov.s@gmail.com>
a
>> écrit
>> >>> :
>> >>> >
>> >>> > Hi,
>> >>> >
>> >>> > Did you turn on cache metrics for your data region?
>> >>> >
>> >>> > To turn the metrics on, use one of the following approaches:
>> >>> > 1. Set DataRegionConfiguration.setMetricsEnabled(true) for every
>> >>> region you
>> >>> > want to collect the metrics for.
>> >>> > 2. Use the DataRegionMetricsMXBean.enableMetrics() method exposed
>> by a
>> >>> > special JMX bean.
>> >>> >
>> >>> > More information regarding cache metrics available here:
>> >>> > https://apacheignite.readme.io/docs/cache-metrics
>> >>> >
>> >>> > Regards,
>> >>> > Igor
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>> >>>
>> >>> This message and any attachments (the "message") is
>> >>> intended solely for the intended addressees and is confidential.
>> >>> If you receive this message in error,or are not the intended
>> >>> recipient(s),
>> >>> please delete it and any copies from your systems and immediately
>> notify
>> >>> the sender. Any unauthorized view, use that does not comply with its
>> >>> purpose,
>> >>> dissemination or disclosure, either whole or partial, is prohibited.
>> >>> Since the internet
>> >>> cannot guarantee the integrity of this message which may not be
>> >>> reliable, BNP PARIBAS
>> >>> (and its subsidiaries) shall not be liable for the message if
>> modified,
>> >>> changed or falsified.
>> >>> Do not print this message unless it is necessary, consider the
>> >>> environment.
>> >>>
>> >>>
>> >>>
>> ----------------------------------------------------------------------------------------------------------------------------------
>> >>>
>> >>> Ce message et toutes les pieces jointes (ci-apres le "message")
>> >>> sont etablis a l'intention exclusive de ses destinataires et sont
>> >>> confidentiels.
>> >>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>> >>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>> >>> avertir
>> >>> immediatement l'expediteur. Toute lecture non autorisee, toute
>> >>> utilisation de
>> >>> ce message qui n'est pas conforme a sa destination, toute diffusion
ou
>> >>> toute
>> >>> publication, totale ou partielle, est interdite. L'Internet ne
>> >>> permettant pas d'assurer
>> >>> l'integrite de ce message electronique susceptible d'alteration, BNP
>> >>> Paribas
>> >>> (et ses filiales) decline(nt) toute responsabilite au titre de ce
>> >>> message dans l'hypothese
>> >>> ou il aurait ete modifie, deforme ou falsifie.
>> >>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>> >>>
>> >>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message