From dev-return-45886-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Mon Apr 29 08:24:10 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 8E66118067E for ; Mon, 29 Apr 2019 10:24:10 +0200 (CEST) Received: (qmail 51662 invoked by uid 500); 29 Apr 2019 08:24:09 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 51603 invoked by uid 99); 29 Apr 2019 08:24:09 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 29 Apr 2019 08:24:09 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 0D6DD18006B for ; Mon, 29 Apr 2019 08:24:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.065 X-Spam-Level: ** X-Spam-Status: No, score=2.065 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FROM_EXCESS_BASE64=0.105, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URI_HEX=1.313, URI_TRY_3LD=0.596] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id h6ggMD8ktrjO for ; Mon, 29 Apr 2019 08:24:07 +0000 (UTC) Received: from mail-ot1-f54.google.com (mail-ot1-f54.google.com [209.85.210.54]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id DF0165F576 for ; Mon, 29 Apr 2019 08:14:37 +0000 (UTC) Received: by mail-ot1-f54.google.com with SMTP id 77so1682096otu.13 for ; Mon, 29 Apr 2019 01:14:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=z1GCIrdthAMRwVNkHjWqd5gg+Ii3G5WRyG+y8dOh+sU=; b=C/1E3psevj9Ij9VVRVlh5wkhNZT3sCMxGFuOeWiImi1jjzZCGL695rMJA1QxBq31H2 ijIjoGTEPOYgcTUKfvXm3rS4KA1IF9q5HxbTyCrM5Zy0XRnshXArr189yFLRSm0a39AR +YKD7/xYO2khpyBMzYDHxzhajYCdyAcOsHLqrSziJaVwFAqKjPnVslKoWWwDMzo4uVPG F2fao/rFDMEUnZjF1LosDHre8I/tZbbXxoVIiQdd7rm92MyVCBx2NKxgJw6IEYqY+alG f5+1utH9nFywSGEIsUWYhyeSvUd1T8WOCbYPjwRowEQcgOKI3cJmIeAzwoCA48+umn73 6+zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=z1GCIrdthAMRwVNkHjWqd5gg+Ii3G5WRyG+y8dOh+sU=; b=b3hK2QpABDQk5k6d6b/+gc7kljhTRbL4YHrEtk0wKmPFQBqD8NodxuNCHFx6vCyhfm E5RJCAH/0INX9/5s52v5lIAtU9yAYCP8AJb1wh042WH03cUa6whgXXeIRx+2GCRgJqSw 6G5poJ7Atu4wnd3Jy+hRXOEecmK68yJEk+dwxyWUiHBEgBoobUvW4xI7wAuCO+vu1Yz4 henYqtywrfLgMmtzfr1jN5+htJt+RiUljbl9II8vfUb+iL+bPf6aEgKU3ewOoczgEVzD 0x5WV/xRXKLvO2C/xHJVUdRa/sHAoE7kYVKSrSPCe+TLyQTBZmu0JH4U4o2rS/KMnRjk zaPw== X-Gm-Message-State: APjAAAVeZbL0u0jKUxjcnf2+IgZBfpTAxdI39g6b5WmaU5wV6gAy8K0y M+bxIUj0Ad+30htqpee6KQ+1Se6z9fa/pRv4Ac9KLVCrEbo= X-Google-Smtp-Source: APXvYqw33dxvEvDwMNvcAKQxwt0FRiVlMtBqlWSYesZxU5bNVEbjhjEOq0jl9m9UKTQkAj994UTzOAUPOtia0g0a7q8= X-Received: by 2002:a9d:6152:: with SMTP id c18mr48157otk.230.1556525676545; Mon, 29 Apr 2019 01:14:36 -0700 (PDT) MIME-Version: 1.0 References: <1543243413308-0.post@n4.nabble.com> In-Reply-To: From: =?UTF-8?B?0J/QsNCy0LvRg9GF0LjQvSDQmNCy0LDQvQ==?= Date: Mon, 29 Apr 2019 11:14:25 +0300 Message-ID: Subject: Re: Brainstorm: Make TC Run All faster To: dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Vyacheslav, Maxim, Can we once again outline what benefits aggregated "Build Apache Ignite" performing various checks has comparing to a modularized approach in which separate builds perform separate tasks? For example, modularized approach looks nice because it is similar to good practices in software development where we separate responsibilities between different classes instead of aggregating them into a single class. And as usual multiple classes works together coordinating by a class from upper level. So, in fact it is a hierarchical structure. Returning to "Build Apache Ignite" it seems to me that ideally it can be hierarchical. There is a top level compilation (assembly?) job but it is always clear what tasks does it perform (check style, check license and other subjobs). =D0=BF=D1=82, 26 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 17:06, Maxim Muzaf= arov : > > Folks, > > +1 for merging all these suites into the single one. All these suites > (Build Apache Ignite, Javadoc, Licenses Header, Checkstyle) required > to be `green` all the time. So, we can consider making them a part of > build Apache Ignite procedure. > > Also, I'd suggest going deeper. We can try to merge `Licenses Header` > into the `Code style checker` [1]. This will simplify the code > checking process. > > [1] http://checkstyle.sourceforge.net/config_header.html > > On Fri, 26 Apr 2019 at 13:17, Vyacheslav Daradur wr= ote: > > > > Ivan, you are right, I meant to combine them into one. > > > > Here is a build [1], with enabled profiles (check-licenses, > > checkstyle) and check of javadoc to show the idea. > > > > Seems it takes ~15 minutes. > > > > [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=3DIgniteTest= s24Java8_ExperimentalBuildApacheIgniteJavadocLicensesHeaderCheckstyle&branc= h_IgniteTests24Java8=3D > > > > On Fri, Apr 26, 2019 at 12:06 PM =D0=9F=D0=B0=D0=B2=D0=BB=D1=83=D1=85= =D0=B8=D0=BD =D0=98=D0=B2=D0=B0=D0=BD wrote: > > > > > > Hi Vyacheslav, > > > > > > What do you mean by uniting? > > > > > > For me it looks like [Javadocs] and [Check Code Style] are not so tim= e > > > consuming comparing to tests, are not they? Do you suggest to combine > > > mentioned 4 jobs into one? How long will it run in a such case? > > > > > > =D1=87=D1=82, 25 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 10:50, Vyach= eslav Daradur : > > > > > > > > Hi Igniters, > > > > > > > > At the moment we have several separated test suites: > > > > * ~Build Apache Ignite~ _ ~10..20mins > > > > * [Javadocs] _ ~10mins > > > > * [Licenses Headers] _ ~1min > > > > * [Check Code Style] _ ~7min > > > > The most time of each build (except Licenses Headers) is taken by > > > > dependency resolving. > > > > > > > > Their main goal is a check that the project is built properly. > > > > > > > > Also, profiles of [Javadocs], [Licenses Headers] uses at the step o= f > > > > preparing release (see DEVNOTES.txt) that means they are important. > > > > > > > > I'd suggest uniting the builds, this should reduce the time of test= s > > > > on ~15 minutes and releases agents. > > > > > > > > What do you think? > > > > > > > > On Tue, Nov 27, 2018 at 3:56 PM =D0=9F=D0=B0=D0=B2=D0=BB=D1=83=D1= =85=D0=B8=D0=BD =D0=98=D0=B2=D0=B0=D0=BD wrote: > > > > > > > > > > Roman, > > > > > > > > > > Do you have some expectations how faster "correlated" tests > > > > > elimination will make Run All? Also do you have a vision how can = we > > > > > determine such "correlated" tests, can we do it relatively fast? > > > > > > > > > > But all in all, I am not sure that reducing a group of correlated > > > > > tests to only one test can show good stability. > > > > > =D0=BF=D0=BD, 26 =D0=BD=D0=BE=D1=8F=D0=B1. 2018 =D0=B3. =D0=B2 17= :48, aplatonov : > > > > > > > > > > > > It should be noticed that additional parameter TEST_SCALE_FACTO= R was added. > > > > > > This parameter with ScaleFactorUtil methods can be used for tes= t size > > > > > > scaling for different runs (like ordinary and nightly RunALLs).= If someone > > > > > > want to distinguish these builds he/she can apply scaling metho= ds from > > > > > > ScaleFactorUtil in own tests. For nightly test TEST_SCALE_FACTO= R=3D1.0, for > > > > > > non-nightly builds TEST_SCALE_FACTOR<1.0. For example in > > > > > > GridAbstractCacheInterceptorRebalanceTest test ScaleFactorUtil = was used for > > > > > > scaling count of iterations. I guess that TEST_SCALE_FACTOR sup= port will be > > > > > > added to runs at the same time with RunALL (nightly) runs. > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.co= m/ > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > -- > > > > Best Regards, Vyacheslav D. > > > > > > > > > > > > -- > > > Best regards, > > > Ivan Pavlukhin > > > > > > > > -- > > Best Regards, Vyacheslav D. --=20 Best regards, Ivan Pavlukhin