ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: Memory usage per cache
Date Wed, 04 Apr 2018 12:27:41 GMT
Denis,

I think this particular metric should be deprecated. The most we can do
about it is to return the actual allocated size when a cache is the only
cache in a group and return -1 if there are multiple caches in a group.
However, this does not look like a consistent approach to me, so I would
prefer to always return -1 and suggest that users use corresponding cache
group metrics.

--AG

2018-03-30 20:04 GMT+03:00 Denis Magda <dmagda@apache.org>:

> Dmitriy G., Alexey,
>
> What happens with CacheMetrics.getOffHeapAllocatedSize metric? Don't see
> it
> mentioned in the tickets.
>
>    - Will it return a value of a respective cache group if the cache is the
>    only one in the group.
>    - What will it return if there are several caches in a group?
>
> --
> Denis
>
> On Fri, Mar 30, 2018 at 8:16 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > Yes, I think we can maintain this metric inside lock ownership change
> > callbacks.
> >
> > 2018-03-30 17:34 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
> >
> > > I think "getLockedKeysCount" is better. Can we easily return this count
> > > without traversing all the transactions?
> > >
> > >
> > >
> > > On Fri, Mar 30, 2018 at 7:32 AM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com> wrote:
> > >
> > > > Dmitriy,
> > > >
> > > > getLockedKeys returns only the number of locked keys, not the keys
> > > > themselves. Looks like the method name is confusing, should we rename
> > it
> > > to
> > > > getLockedKeysNumber?
> > > >
> > > > --AG
> > > >
> > > > 2018-03-30 16:24 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org
> >:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > For transactions, are these separate metrics for each transaction
> or
> > > > > aggregates for all of them? If these are aggregates, then methods
> > like
> > > > > "getLockedKeys" may produce a very large collection, which on top
> of
> > > > > everything else, will be very difficult to create.
> > > > >
> > > > > D.
> > > > >
> > > > > On Fri, Mar 30, 2018 at 2:04 AM, Dmitriy Govorukhin <
> > > > > dmitriy.govorukhin@gmail.com> wrote:
> > > > >
> > > > > > Folks,
> > > > > >
> > > > > > I created 2 Jira issue for new metrics. Also, I think they will
> be
> > > > > useful.
> > > > > > Let's discuss in comments.
> > > > > >
> > > > > > - Add new metrics for data storage IGNITE-8078
> > > > > > <https://issues.apache.org/jira/browse/IGNITE-8078>
> > > > > > - Implement new JMX metrics for transactions IGNITE-8077
> > > > > > <https://issues.apache.org/jira/browse/IGNITE-8077>
> > > > > >
> > > > > > On Thu, Mar 29, 2018 at 8:26 PM, Denis Magda <dmagda@apache.org>
> > > > wrote:
> > > > > >
> > > > > > > Alexey,
> > > > > > >
> > > > > > > If to rephrase it differently, if a cache/table belongs
to a
> > single
> > > > > cache
> > > > > > > group then its entries will never be mixed with the entries
of
> > > > another
> > > > > > > caches/tables in both data and index pages. Please confirm
that
> > my
> > > > > > > statement is correct. Also, does that counter accumulates
size
> of
> > > > > > > BinaryObjects or the whole data/index pages?
> > > > > > >
> > > > > > > Finally, do you plan to rework CacheMetrics.
> > > getOffHeapAllocatedSize
> > > > > > method
> > > > > > > for this purpose? (btw, to my knowledge the method is broken
at
> > the
> > > > > > moment
> > > > > > > and returns 0).
> > > > > > >
> > > > > > > --
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Mar 29, 2018 at 1:20 AM, Alexey Goncharuk <
> > > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > > >
> > > > > > > > Denis,
> > > > > > > >
> > > > > > > > Currently there is an easy way to add per-cache-group
metrics
> > for
> > > > > data
> > > > > > > and
> > > > > > > > index pages. There is an internal counter, but it
is not
> > > published
> > > > as
> > > > > > an
> > > > > > > > MBean metric, we will do this as a part of IEP-6.
> > > > > > > >
> > > > > > > > As for the per-cache metrics, this can be implemented,
but it
> > > will
> > > > > > take a
> > > > > > > > significantly greater effort and most likely will
affect
> > > > performance.
> > > > > > As
> > > > > > > > Andrey noted, when two caches share the same cache
group,
> > > multiple
> > > > > > > entries
> > > > > > > > may be written to the same page, this needs special
> handling. I
> > > > > suggest
> > > > > > > we
> > > > > > > > first start with per-cache-group metrics and then,
if there
> is
> > > high
> > > > > > > demand,
> > > > > > > > start thinking about per-cache metrics.
> > > > > > > >
> > > > > > > > --AG
> > > > > > > >
> > > > > > > > 2018-03-27 15:18 GMT+03:00 Andrey Kuznetsov <
> stkuzma@gmail.com
> > >:
> > > > > > > >
> > > > > > > > > I apologize for the previous message sent in
hurry. It's
> > > > imposible
> > > > > to
> > > > > > > > > measure the difference between 'precise' and
'estimated'
> page
> > > > > memory
> > > > > > > > usage
> > > > > > > > > per cache unless we fully implement approach
2. Current
> model
> > > > > allows
> > > > > > to
> > > > > > > > > store objects from several caches in a single
page.
> > > > > > > > >
> > > > > > > > > 2018-03-23 22:21 GMT+03:00 Andrey Kuznetsov <
> > stkuzma@gmail.com
> > > >:
> > > > > > > > >
> > > > > > > > > > Denis,
> > > > > > > > > >
> > > > > > > > > > I'll need to conduct some experiments to
estimate the
> > > > difference.
> > > > > > And
> > > > > > > > the
> > > > > > > > > > answer will depend on numerous parameters:
object sizes,
> > > number
> > > > > of
> > > > > > > > > caches,
> > > > > > > > > > that share the same data region and so on.
> > > > > > > > > >
> > > > > > > > > > пт, 23 марта 2018, 21:53 Denis Magda
<dmagda@apache.org
> >:
> > > > > > > > > >
> > > > > > > > > >> Andrey,
> > > > > > > > > >>
> > > > > > > > > >> How good will be the estimate if we
go for 1. and
> utilize
> > > > > > > > > pagesFillFactor
> > > > > > > > > >> somehow? In other words, how big can
be a difference
> > between
> > > > > 100%
> > > > > > > > > precise
> > > > > > > > > >> calculation you the approach you're
suggesting?
> > > > > > > > > >>
> > > > > > > > > >> --
> > > > > > > > > >> Denis
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > >   Andrey Kuznetsov.
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message