geode-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincent Ford <vf...@pivotal.io>
Subject Re: [Discussion] Geode-Native Removing Stats from Public API
Date Tue, 21 Nov 2017 18:25:17 GMT
I would have some concerns with reducing the availability of stats or
devaluing the tooling for analysis. Although exposing more stats via JMX is
a great idea, most of the third party tooling I have seen is best for
monitoring and not post analysis. Not investing in jVSD seems like it would
make this post crash/issue analysis much more difficult in reviewing the
complex interactions among multiple distributed nodes. For example
correlating performance issues among 10 JVM's ( ie a GC in one node causing
performance hiccups detected in another node) and be extremely difficult
without using a tool that can consume and allow the overlay of multiple
graphs of metrics we are collecting from many sources.  The power here is
in the ability to correlate multiple events from multiple JVM's into a
single graphical view for debugging purposes and without this capability it
will be significantly more difficult to understand the complex distributed
behavior of Geode.

Currently custom stats are basically undocumented and difficult to use,
removing them from the public API will probably have little impact on  most
users. Most users that want to do some monitoring can use JMX for their
compentents but it would be helpful to have a method to add those values
into the same stream as other stats/metrics for post issue analysis.



*Vince Ford*
GemFire Toolsmith Engineering
Beaverton, OR USA
http://www.pivotal.io
Open Source Project Geode http://geode.apache.org/
<https://network.pivotal.io/products/project-geode>

On Mon, Nov 20, 2017 at 11:46 PM, Jacob Barrett <jbarrett@pivotal.io> wrote:

> Couldn’t agree more for the Java side of things. The first step is
> deprecating the API for adding custom stats.
>
> > On Nov 20, 2017, at 11:13 PM, Swapnil Bawaskar <sbawaskar@pivotal.io>
> wrote:
> >
> > A lot of statistics we have are exposed over JMX. I think we should make
> an
> > effort to expose all the stats over JMX, making them consumable with
> > existing tooling rather than investing in jVSD.
> >
> >> On Mon, Nov 20, 2017 at 2:32 PM Addison Huddy <ahuddy@pivotal.io>
> wrote:
> >>
> >> Thanks for the clarification Jake. So yes, I'm in agreement that we
> >> should simplify the C++ API and remove the stats API from the C++
> client.
> >>
> >> \ah
> >>
> >> On Mon, Nov 20, 2017 at 10:23 AM, Jacob Barrett <jbarrett@pivotal.io>
> >> wrote:
> >>
> >>> To clarify, the goal here is to remove access from the public API to
> >> inject
> >>> custom stats into our stats stream. We would still collect stats for
> the
> >>> internals of the client.
> >>>
> >>> The rational is multifaceted:
> >>>
> >>> 1) The C++ API is would need a non-trivial amount of time to modernize.
> >> The
> >>> current API uses pointers of pointers for maintaining collections.
> There
> >> is
> >>> a strange cross relationship between components in the stats classes
> >> which
> >>> create unique pointer ownership questions. Rather than solving those
> now
> >>> and further dragging out the modernization of the C++ API we would drop
> >> it
> >>> and evaluated adding it back in a modern way in the future. Though I
> >>> suspect adding it back in the future will never happen for the reasons
> >>> below.
> >>>
> >>> 2) The storage format for our stats in proprietary to Geode and lacks
> >> wide
> >>> market adoption.
> >>>
> >>> 3) There are no modern tools for analyzing the statistics generated.
> >> Geode
> >>> lacks a tool for viewing or analyzing the statistics. Unless work is
> >>> prioritized on completing the jVSD application then users are forced to
> >>> write custom applications to extract the contents of the stats files.
> >>>
> >>> I support the removal from the Java public API for reasons 2 and 3.
> >> Unless
> >>> we put a full effort into creating the ecosystem around the stats
> format
> >> to
> >>> make it usable we should remove it from the public API. I strongly
> >>> encourage that we remove it internally too but that is for another
> >>> discussion.
> >>>
> >>> -Jake
> >>>
> >>>
> >>>> On Mon, Nov 20, 2017 at 9:43 AM Michael Stolz <mstolz@pivotal.io>
> wrote:
> >>>>
> >>>> I'm not clear on why we are removing stats gathering capability.
> >>>> Do we know that customers aren't using this?
> >>>> Is it badly broken?
> >>>>
> >>>> What is actually driving this work?
> >>>>
> >>>> --
> >>>> Mike Stolz
> >>>> Principal Engineer, GemFire Product Lead
> >>>> Mobile: +1-631-835-4771 <(631)%20835-4771> <(631)%20835-4771>
> >>>>
> >>>> On Mon, Nov 20, 2017 at 11:42 AM, Bruce Schuchardt <
> >>> bschuchardt@pivotal.io
> >>>>>
> >>>> wrote:
> >>>>
> >>>>> Should this be done for the Java caches as well?
> >>>>>
> >>>>>
> >>>>>> On 11/17/17 11:48 AM, David Kimura wrote:
> >>>>>>
> >>>>>> I agree, a statistics interface seems beyond the scope of Geode
> >> Native
> >>>>>> client responsibility.  Hiding or removing seems appropriate
to me.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> David
> >>>>>>
> >>>>>> On Fri, Nov 17, 2017 at 11:29 AM, Ernest Burghardt
> >>>>>> <eburghardt@pivotal.io> wrote:
> >>>>>>
> >>>>>>> +1 for removal
> >>>>>>>
> >>>>>>> On Thu, Nov 16, 2017 at 1:46 PM, Jacob Barrett <
> >> jbarrett@pivotal.io>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> I want to open a discussion regarding the removal of
> >>> StatisticsFactory
> >>>>>>>> and
> >>>>>>>> related APIs from the public API. I can't see that we
would want
> >> the
> >>>>>>>> Geode
> >>>>>>>> Native client to be a first class statistics/metrics
gathering
> >> API.
> >>>>>>>> There
> >>>>>>>> are plenty of other first class players in this space.
If this
> >>> isn't a
> >>>>>>>> feature of the client then I suggest it be moved internally.
It’s
> >>>> highly
> >>>>>>>> unlikely it’s being used but in the case that it is
we can
> >> consider
> >>>>>>>> moving
> >>>>>>>> it back after some serious refactoring as it relies
on an over
> >>>>>>>> abundance of
> >>>>>>>> raw pointers. Rather than spend time refactoring it
now let’s just
> >>>> hide
> >>>>>>>> it
> >>>>>>>> away.
> >>>>>>>>
> >>>>>>>> -Jake
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>
> >>>>
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message