From dev-return-43743-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Tue Dec 25 12:50:17 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 2552B180645 for ; Tue, 25 Dec 2018 12:50:16 +0100 (CET) Received: (qmail 11108 invoked by uid 500); 25 Dec 2018 11:50:16 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 11096 invoked by uid 99); 25 Dec 2018 11:50:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Dec 2018 11:50:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 18935C00CE for ; Tue, 25 Dec 2018 11:50:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.111 X-Spam-Level: *** X-Spam-Status: No, score=3.111 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URI_HEX=1.313] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 9QdN_u7sO9da for ; Tue, 25 Dec 2018 11:50:13 +0000 (UTC) Received: from mail-vs1-f51.google.com (mail-vs1-f51.google.com [209.85.217.51]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id E9A8D5F535 for ; Tue, 25 Dec 2018 11:50:12 +0000 (UTC) Received: by mail-vs1-f51.google.com with SMTP id t17so8414604vsc.8 for ; Tue, 25 Dec 2018 03:50:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=pNdr2TH8l6il1C7xPHXBkMYtR7Di6OUWRx6WBiVEK8o=; b=dA0nj/w5BY6gYtGF7dyH3VyhtbSRshy1OfIrRFfhvGpgLTSQW6M0IFsrf0hg6UIFg/ pKppVrW/Yq+H4Qyg75CwbauGv8lok4ZA7C4dD7ACnxIKkaW3K0+NKKDfBR3eK5trqcZ4 1VSEhylU/EL3+n6s1LkhNGJZXO4X/EHjiT3irGS2QZl3LQsn9u6Qru9FU4Uf3GkqvpZy KDvCNOGK7pe3wN2zCzs38sNoX1uHC4I4znBAdFA7zGtmco4FFLIeFAXECl8PtiY/SD1+ C9pZG0YWidXgPSwBHJYNecXa+Avz2yfb0JDvnqpjcG5SRim1OpfyFuVK5pKXs6NPUIjm dfKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=pNdr2TH8l6il1C7xPHXBkMYtR7Di6OUWRx6WBiVEK8o=; b=lSjx2ChpcbU/k4uznzjZ5Z1an4nXIrDuawvVHHThX8C6wyvxvJXr85wQozHXODy+hh X/hof68UTER+gF9hA1XQY5MJ85dO7HFib5zYihlFZlDW7OTyXCw24HyzP51PzRhSpBJF ZT3wCPevkhhYGiVGJwN1W39TUTjW0zWtDWFM7kaKNiw1ghu1fvZJGOn5+m+77sVtGxRG 66Ynp9GIDjQaOLwxMYtBGSyWXfjjhXYRnwfqYR1Oig16413jp6dlKNManjVTDvz+RIG+ atEBOssTy6yYKshJcFiCPVOaEec1BexLTcOc2HPJ9UEpaVKYp4ejYXRefAMD8F+akohw lKyg== X-Gm-Message-State: AA+aEWa4IV2Yn55EgyKIDciE8vVH51xbhUmSlUgVqjxNMtlg/Ps+18hZ UKEClxWjMR5yzLeXsrsQ/RLxu4hJTt/qHQDFodSeuqOL X-Google-Smtp-Source: AFSGD/V38cTDzV+dLHfgF8op1dfcPTh1TbsmqVuAthaXSkTJK9N3WoKvhpM+XApJXbEIrnihs0orGuiTG2XdBeNxqaI= X-Received: by 2002:a67:5b01:: with SMTP id p1mr6054135vsb.132.1545738612137; Tue, 25 Dec 2018 03:50:12 -0800 (PST) MIME-Version: 1.0 References: <1545737054733-0.post@n4.nabble.com> In-Reply-To: <1545737054733-0.post@n4.nabble.com> From: Andrey Mashenkov Date: Tue, 25 Dec 2018 14:50:00 +0300 Message-ID: Subject: Re: Apache Ignite TeamCity Bot - muted tests To: dev Content-Type: multipart/alternative; boundary="000000000000e4628d057dd750c4" --000000000000e4628d057dd750c4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Oleg, Thanks for clarification. On Tue, Dec 25, 2018 at 2:24 PM oignatenko wrote: > Hi Andrey, > > As far as I know tests annotated by @Ignore can be conveniently tracked o= n > Teamcity but this requires such tests to be included into JUnit 4 test > suite > (org.junit.runners.Suite). > > Above requirement currently isn't met by many Ignite tests which are > included in JUnit 3 suites (junit.framework.TestSuite). This means that > test > cases annotated @Ignore simply are hidden when present in such > old-fashioned > suites. > > We are currently working on migrating Junit 3 suites to JUnit 4. If you a= re > interested, work breakdown and progress can be found in JIRA: > - https://issues.apache.org/jira/browse/IGNITE-10762 > > ----- > > With regards to Assume API, it seems more complicated. I have recently re= ad > that tests that are conditionally skipped using this API are reported by > JUnit 4 test framework as passed. This is not very convenient. > > I think I have seen somewhere mentioned that proper reporting of > conditionally skipped tests can be achieved only with JUnit 5. But I didn= 't > dig deeper there because we haven't yet decided whether it is worth it fo= r > Ignite to move to JUnit 5 > (https://issues.apache.org/jira/browse/IGNITE-10180). > > Given above, I believe that the proper way to mute tests is currently wit= h > @Ignore and Assume API. It's not 100% ideal but overall looks better than > the old one. > > regards, Oleg > > > Andrew Mashenkov wrote > > Dmitry, > > > > As far as I understand, @IgniteIgnore and fail("") works in similar way > > and > > requires test to be muted on TC manually. > > And we have to unmute these tests on TC right after fix will be merged = to > > master, isn't it? > > > > @Ignore and Assume.that works different way comparing to previous two. > > Junit just exclude such tests from run. > > So, there is no need to mute\unmute them on TC at all, but... > > Does TC treat such tests as muted ones and we can see them on "muted > > tests" > > page? > > This may be useful to track tests state if someone forget to remove > > @ignore > > while fixing related ticket. > > > > > > On Mon, Dec 17, 2018 at 5:54 PM Anton Vinogradov < > > > av@ > > > > wrote: > > > >> >> In this case, we will not need TeamCity mutes anymore. > >> Sounds great! > >> > >> On Mon, Dec 17, 2018 at 5:23 PM Dmitrii Ryabov < > > > somefireone@ > > > > > >> wrote: > >> > >> > Dmitriy, > >> > > >> > with ignore annotations we will need to check forgotten tests > >> > manually. Thats bad, but we will save some machine time in common > >> > runs. > >> > =D0=BF=D0=BD, 17 =D0=B4=D0=B5=D0=BA. 2018 =D0=B3. =D0=B2 17:06, Dmit= riy Pavlov < > > > dpavlov@ > > > >: > >> > > > >> > > Investigation parameters: Can leave as empty > >> > > Unmute: Manual (don't use auto because of flaky tests) > >> > > Text: https://issues.apache.org/jira/browse/IGNITE-NNNNN > >> > > > >> > > But anyway, I hope in the nearest future best way to mute test wil= l > >> be > >> to > >> > > ignore it using: > >> > > @IgniteIgnore > >> > > or @Ignore > >> > > or Assume.that() > >> > > > >> > > In this case, we will not need TeamCity mutes anymore. > >> > > > >> > > I'm not sure which way of ignoring will be best. > >> > > > >> > > =D0=BF=D0=BD, 17 =D0=B4=D0=B5=D0=BA. 2018 =D0=B3. =D0=B2 17:02, An= ton Vinogradov < > > > av@ > > > >: > >> > > > >> > > > Dmitrii, > >> > > > > >> > > > My question was mostly about how to mute test properly. > >> > > > > >> > > > > >> > > > On Mon, Dec 17, 2018 at 4:27 PM Dmitrii Ryabov < > >> > > > somefireone@ > > >> > >> > > > wrote: > >> > > > > >> > > > > Anton, > >> > > > > > >> > > > > In the "Mute" column you can find link to the test on TeamCity= . > >> On > >> > > > > this page you can see test details like run history and curren= t > >> > mutes. > >> > > > > Here you should be able to unmute existing mutes. > >> > > > > =D0=BF=D0=BD, 17 =D0=B4=D0=B5=D0=BA. 2018 =D0=B3. =D0=B2 16:21= , Anton Vinogradov < > > > av@ > > > >: > >> > > > > > > >> > > > > > Great feature! > >> > > > > > > >> > > > > > Could you please explain how to have a deal with it? > >> > > > > > How should I mite/unmute tests for now? > >> > > > > > > >> > > > > > On Fri, Dec 14, 2018 at 6:50 PM Dmitriy Pavlov < > >> > > > dpavlov@ > > >> > > > >> > > > > wrote: > >> > > > > > > >> > > > > > > We have IgniteIgnore, so we can use it. It seems behavior = is > >> the > >> > > > same. > >> > > > > We > >> > > > > > > need just to understand in which case we can get issue ID > >> from > >> TC > >> > > > REST > >> > > > > (for > >> > > > > > > the Bot). > >> > > > > > > > >> > > > > > > =D0=BF=D1=82, 14 =D0=B4=D0=B5=D0=BA. 2018 =D0=B3. =D0=B2 1= 7:51, Vyacheslav Daradur < > >> > > > > daradurvs@ > > >> > > > >: > >> > > > > > > > >> > > > > > > > Dmitry Pavlov, > >> > > > > > > > > >> > > > > > > > As far as I know, we are able to use following > construction > >> to > >> > > > ignore > >> > > > > > > test: > >> > > > > > > > Assume.assumeTrue("link-to-issue", false); > >> > > > > > > > > >> > > > > > > > On Fri, Dec 14, 2018 at 5:44 PM Dmitriy Pavlov < > >> > > > > dpavlov@ > > >> > > > > > >> > > > > > > wrote: > >> > > > > > > > > > >> > > > > > > > > Thank you, Dmitrii, > >> > > > > > > > > > >> > > > > > > > > I've started unmuting tests starting from yesterday. > >> > > > > > > > > > >> > > > > > > > > Now muted test count was decreased from 3540 to 3408 > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > >> > https://ci.ignite.apache.org/project.html?projectId=3DIgniteTests24Java8&= tab=3DmutedProblems&branch_IgniteTests24Java8=3D%3Cdefault%3E > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > Probably we could automate this activity by auto-unmut= e, > >> but > >> > > > > hopefully, > >> > > > > > > > > JUnit4 & @Ignore opportunity will come earlier. > >> > > > > > > > > > >> > > > > > > > > Sincerely, > >> > > > > > > > > Dmitriy Pavlov > >> > > > > > > > > > >> > > > > > > > > =D0=BF=D1=82, 14 =D0=B4=D0=B5=D0=BA. 2018 =D0=B3. =D0= =B2 17:24, Dmitrii Ryabov < > >> > > > > > > somefireone@ > > >> > > > > >: > >> > > > > > > > > > >> > > > > > > > > > Hello, Igniters! > >> > > > > > > > > > > >> > > > > > > > > > Our Bot got new functionality - it shows tests muted > on > >> > > > TeamCity > >> > > > > and > >> > > > > > > > their > >> > > > > > > > > > ticket status [1]. > >> > > > > > > > > > > >> > > > > > > > > > So, we can see forgotten mutes for resolved tickets > >> now. > >> > > > > > > > > > > >> > > > > > > > > > Igniters with rights to unmute tests on TC - please, > >> check > >> > this > >> > > > > page > >> > > > > > > > > > periodically for forgotten mutes. > >> > > > > > > > > > > >> > > > > > > > > > [1] https://mtcga.gridgain.com/mutes.html > >> > > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > -- > >> > > > > > > > Best Regards, Vyacheslav D. > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > >> > > > > > > -- > > Best regards, > > Andrey V. Mashenkov > > > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > --=20 Best regards, Andrey V. Mashenkov --000000000000e4628d057dd750c4--