From dev-return-36405-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Tue Jul 10 19:02:17 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id E9344180663 for ; Tue, 10 Jul 2018 19:02:16 +0200 (CEST) Received: (qmail 64253 invoked by uid 500); 10 Jul 2018 17:02:16 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 64242 invoked by uid 99); 10 Jul 2018 17:02:15 -0000 Received: from mail-relay.apache.org (HELO mailrelay1-lw-us.apache.org) (207.244.88.152) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Jul 2018 17:02:15 +0000 Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com [209.85.218.46]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 04EF86F1 for ; Tue, 10 Jul 2018 17:02:14 +0000 (UTC) Received: by mail-oi0-f46.google.com with SMTP id r16-v6so44004213oie.3 for ; Tue, 10 Jul 2018 10:02:14 -0700 (PDT) X-Gm-Message-State: APt69E1qppEynTXPriWgFApqQ/DT/9TJzG0Os6KCb843bwwSW0kjyN6h AcOzyb5qFJ0hROsXZKBhyReU7xzc6nnpyltZHbytdw== X-Google-Smtp-Source: AAOMgpc242+CNATvUGP6XJWkVlbYdX1Equa3M/hq45fZWqPUifVPTsRxaN3US7mPLsbbMETSmjAdFnTRJGCaLHjMyts= X-Received: by 2002:aca:c40e:: with SMTP id u14-v6mr26587391oif.288.1531242134331; Tue, 10 Jul 2018 10:02:14 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ac9:3362:0:0:0:0:0 with HTTP; Tue, 10 Jul 2018 10:01:33 -0700 (PDT) In-Reply-To: References: <0a207b9e0d784552725090a97ebd47cea2ee0596.camel@gmail.com> <54ae81bf-d794-6dad-2cfb-6f13cb41edf8@gmail.com> From: Dmitriy Setrakyan Date: Tue, 10 Jul 2018 19:01:33 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Clusterwide settings validation To: dev Content-Type: multipart/alternative; boundary="0000000000007b758c0570a817c6" --0000000000007b758c0570a817c6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable What about an idea to have a validation template file (e.g. ignite-validate.xml), and make sure on startup that all config properties specified in that file match. This way a user could put this file somewhere on a shared network drive and have an extra degree of confidence that the configuration is valid. Thoughts? D. On Tue, Jul 10, 2018 at 5:43 PM, Ivan Rakov wrote: > Slava, > > I agree. Different persistence enabled flag can cause unpleasant issues. > I've left a comment in IGNITE-8951. > > Yakov, > > Seems like I misunderstood the point of the discussion from the very > beginning. I thought that Andrew raised topic to discuss adding new check= s > that will fail node join (like we do for different page size and rebalanc= e > pool size). If we are talking about /printing warnin//gs//about all > differences/, we indeed can start with logic that passes through > configuration classes with reflection. As a next step, we can filter out > the properties that are expected to be different (consistentId, etc). I > believe, full list of such properties can't be collected without manual > research. > > Best Regards, > Ivan Rakov > > > On 10.07.2018 14:06, =D0=92=D1=8F=D1=87=D0=B5=D1=81=D0=BB=D0=B0=D0=B2 =D0= =9A=D0=BE=D0=BF=D1=82=D0=B8=D0=BB=D0=B8=D0=BD wrote: > >> .Hello Ivan, >> >> I think it would be nice to add validation >> DataRegionConfiguration#persistenceEnabled property. That property must >> be >> the same across a cluster for the given data region. >> Perhaps, different values of `initSize`, `maxSize` etc make sense in cas= e >> of a heterogeneous cluster, except `persistenceEnabled` >> >> Thanks, >> S. >> >> =D0=B2=D1=82, 10 =D0=B8=D1=8E=D0=BB. 2018 =D0=B3. =D0=B2 13:42, Ivan Rak= ov : >> >> Guys, >>> >>> For your information: rebalanceThreadPoolSize validation is already >>> implemented and merged to master: >>> https://issues.apache.org/jira/browse/IGNITE-8904 >>> You can overview the commit to see the approach. By the way, we already >>> validate some other parameters that can't differ among cluster nodes >>> (page size, tx configuration): GridCacheProcessor#checkConsistency. >>> We also match necessary part of CacheConfiguration between nodes in >>> GridCacheUtils#checkAttributeMismatch method. >>> >>> Does anyone know another properties mismatch of which can backfire on u= s? >>> >>> Best Regards, >>> Ivan Rakov >>> >>> On 10.07.2018 10:47, Andrew Medvedev wrote: >>> >>>> Made comment there, c&p here as well >>>> >>>> Is it going to be a preconfigured set of settings, or a whole range >>>> of settings? >>>> >>>> If latter : >>>> >>>> 1) Property names in CacheConfiguration do not always correspond to >>>> getters (some follow different naming conventions, some are completely >>>> different, as in memPlcName and getDataRegionName()), so inclusion >>>> pattern ("get all properties") does not work quite well with them >>>> >>>> 2) If using manual handling of getter methods, we see that a lot of >>>> metrics are returned by methods in CacheConfiguration and below, >>>> instead of properties (in TcpCommunicationSpi especially), and getter >>>> methods are not properly annotated. (for ex see >>>> https://issues.apache.org/jira/browse/IGNITE-8829), so exclusion >>>> pattern ("get all except metrics etc") forces us to manually exclude >>>> those, not quite well too, looks like a hack >>>> >>>> Plus some properties, although configurable, have their defaults >>>> dynamically set on startup for ex. DFLT_MEMORY_POLICY_MAX_SIZE >>>> >>>> Just to make sure, we compare with coordinator, log locally, and >>>> client nodes are excluded? >>>> >>>> On Fri, Jul 6, 2018 at 4:15 PM, Yakov Zhdanov >>>> >>> wrote: >>> >>>> Guys, I created ticket for config params validation - >>>>> https://issues.apache.org/jira/browse/IGNITE-8951. Feel free to >>>>> >>>> comment. >>> >>>> Yakov Zhdanov >>>>> www.gridgain.com >>>>> >>>>> 2018-07-04 10:51 GMT+03:00 Andrew Medvedev < >>>>> andrew.y.medvedev@gmail.com >>>>> >>>> : >>>> >>>>> Hi Nikolay >>>>>> >>>>>> No, we have been beaten by >>>>>> https://issues.apache.org/jira/browse/IGNITE-8904?jql=3Dtext%20~%20% >>>>>> 22rebalanceThreadPoolSize%22 >>>>>> it is not checked on start >>>>>> >>>>>> Utility I mean check >>>>>> org.apache.ignite.configuration.IgniteConfiguration and children >>>>>> >>>>>> On Wed, Jul 4, 2018 at 10:36 AM, Nikolay Izhikov >>>>> > >>>>>> wrote: >>>>>> >>>>>>> Hello, Andrew. >>>>>>> >>>>>>> Can you clarify your question? >>>>>>> >>>>>>> What checks do you mean, exactly? >>>>>>> Do you mean internal Ignite checks or user-provided checks? >>>>>>> >>>>>>> Ignite checks configuration consistency on node start [1]. >>>>>>> >>>>>>> Ignite do have consistency check for a joining node. Take a look at >>>>>>> >>>>>> [2] >>> >>>> and all of it children. >>>>>> >>>>>>> [1] https://github.com/apache/ignite/blob/master/modules/ >>>>>>> >>>>>> core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L825 >>>>>> >>>>>>> [2] https://github.com/apache/ignite/blob/master/modules/ >>>>>>> >>>>>> core/src/main/java/org/apache/ignite/internal/GridComponent.java#L15= 3 >>>>>> >>>>>>> =D0=92 =D0=A1=D1=80, 04/07/2018 =D0=B2 08:58 +0300, Andrew Medvedev= =D0=BF=D0=B8=D1=88=D0=B5=D1=82: >>>>>>> >>>>>>>> Hello everybody >>>>>>>> >>>>>>>> Our company has lots of nodes in cluster, and we have seen some >>>>>>>> problems with inconsistent settings on nodes clusterwide. To help = us >>>>>>>> with this, we made an utility to check consistency of settings on >>>>>>>> running cluster, but it is a hack, better ways seems to be setting= s >>>>>>>> validation by each node itself on start/joining topology/etc.. >>>>>>>> >>>>>>>> 1) Is his needed? >>>>>>>> 2) Have the implementation details been discussed somewhere? >>>>>>>> >>>>>>>> Cheers >>>>>>>> >>>>>>> >>> > --0000000000007b758c0570a817c6--