ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Memory usage per cache
Date Fri, 30 Mar 2018 17:04:31 GMT
Dmitriy G., Alexey,

What happens with CacheMetrics.getOffHeapAllocatedSize metric? Don't see it
mentioned in the tickets.

   - Will it return a value of a respective cache group if the cache is the
   only one in the group.
   - What will it return if there are several caches in a group?

--
Denis

On Fri, Mar 30, 2018 at 8:16 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> Yes, I think we can maintain this metric inside lock ownership change
> callbacks.
>
> 2018-03-30 17:34 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
>
> > I think "getLockedKeysCount" is better. Can we easily return this count
> > without traversing all the transactions?
> >
> >
> >
> > On Fri, Mar 30, 2018 at 7:32 AM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > Dmitriy,
> > >
> > > getLockedKeys returns only the number of locked keys, not the keys
> > > themselves. Looks like the method name is confusing, should we rename
> it
> > to
> > > getLockedKeysNumber?
> > >
> > > --AG
> > >
> > > 2018-03-30 16:24 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
> > >
> > > > Dmitriy,
> > > >
> > > > For transactions, are these separate metrics for each transaction or
> > > > aggregates for all of them? If these are aggregates, then methods
> like
> > > > "getLockedKeys" may produce a very large collection, which on top of
> > > > everything else, will be very difficult to create.
> > > >
> > > > D.
> > > >
> > > > On Fri, Mar 30, 2018 at 2:04 AM, Dmitriy Govorukhin <
> > > > dmitriy.govorukhin@gmail.com> wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > I created 2 Jira issue for new metrics. Also, I think they will be
> > > > useful.
> > > > > Let's discuss in comments.
> > > > >
> > > > > - Add new metrics for data storage IGNITE-8078
> > > > > <https://issues.apache.org/jira/browse/IGNITE-8078>
> > > > > - Implement new JMX metrics for transactions IGNITE-8077
> > > > > <https://issues.apache.org/jira/browse/IGNITE-8077>
> > > > >
> > > > > On Thu, Mar 29, 2018 at 8:26 PM, Denis Magda <dmagda@apache.org>
> > > wrote:
> > > > >
> > > > > > Alexey,
> > > > > >
> > > > > > If to rephrase it differently, if a cache/table belongs to a
> single
> > > > cache
> > > > > > group then its entries will never be mixed with the entries
of
> > > another
> > > > > > caches/tables in both data and index pages. Please confirm that
> my
> > > > > > statement is correct. Also, does that counter accumulates size
of
> > > > > > BinaryObjects or the whole data/index pages?
> > > > > >
> > > > > > Finally, do you plan to rework CacheMetrics.
> > getOffHeapAllocatedSize
> > > > > method
> > > > > > for this purpose? (btw, to my knowledge the method is broken
at
> the
> > > > > moment
> > > > > > and returns 0).
> > > > > >
> > > > > > --
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Thu, Mar 29, 2018 at 1:20 AM, Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > >
> > > > > > > Denis,
> > > > > > >
> > > > > > > Currently there is an easy way to add per-cache-group metrics
> for
> > > > data
> > > > > > and
> > > > > > > index pages. There is an internal counter, but it is not
> > published
> > > as
> > > > > an
> > > > > > > MBean metric, we will do this as a part of IEP-6.
> > > > > > >
> > > > > > > As for the per-cache metrics, this can be implemented,
but it
> > will
> > > > > take a
> > > > > > > significantly greater effort and most likely will affect
> > > performance.
> > > > > As
> > > > > > > Andrey noted, when two caches share the same cache group,
> > multiple
> > > > > > entries
> > > > > > > may be written to the same page, this needs special handling.
I
> > > > suggest
> > > > > > we
> > > > > > > first start with per-cache-group metrics and then, if there
is
> > high
> > > > > > demand,
> > > > > > > start thinking about per-cache metrics.
> > > > > > >
> > > > > > > --AG
> > > > > > >
> > > > > > > 2018-03-27 15:18 GMT+03:00 Andrey Kuznetsov <stkuzma@gmail.com
> >:
> > > > > > >
> > > > > > > > I apologize for the previous message sent in hurry.
It's
> > > imposible
> > > > to
> > > > > > > > measure the difference between 'precise' and 'estimated'
page
> > > > memory
> > > > > > > usage
> > > > > > > > per cache unless we fully implement approach 2. Current
model
> > > > allows
> > > > > to
> > > > > > > > store objects from several caches in a single page.
> > > > > > > >
> > > > > > > > 2018-03-23 22:21 GMT+03:00 Andrey Kuznetsov <
> stkuzma@gmail.com
> > >:
> > > > > > > >
> > > > > > > > > Denis,
> > > > > > > > >
> > > > > > > > > I'll need to conduct some experiments to estimate
the
> > > difference.
> > > > > And
> > > > > > > the
> > > > > > > > > answer will depend on numerous parameters: object
sizes,
> > number
> > > > of
> > > > > > > > caches,
> > > > > > > > > that share the same data region and so on.
> > > > > > > > >
> > > > > > > > > пт, 23 марта 2018, 21:53 Denis Magda <dmagda@apache.org>:
> > > > > > > > >
> > > > > > > > >> Andrey,
> > > > > > > > >>
> > > > > > > > >> How good will be the estimate if we go for
1. and utilize
> > > > > > > > pagesFillFactor
> > > > > > > > >> somehow? In other words, how big can be a
difference
> between
> > > > 100%
> > > > > > > > precise
> > > > > > > > >> calculation you the approach you're suggesting?
> > > > > > > > >>
> > > > > > > > >> --
> > > > > > > > >> Denis
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > >   Andrey Kuznetsov.
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message