From dev-return-45877-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Sat Apr 27 08:10:53 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id DBE99180630 for ; Sat, 27 Apr 2019 10:10:52 +0200 (CEST) Received: (qmail 31510 invoked by uid 500); 27 Apr 2019 08:10:52 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 31499 invoked by uid 99); 27 Apr 2019 08:10:51 -0000 Received: from Unknown (HELO mailrelay2-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 27 Apr 2019 08:10:51 +0000 Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by mailrelay2-lw-us.apache.org (ASF Mail Server at mailrelay2-lw-us.apache.org) with ESMTPSA id 8CA7837BD for ; Sat, 27 Apr 2019 08:10:51 +0000 (UTC) Received: by mail-qk1-f173.google.com with SMTP id w20so3285363qka.7 for ; Sat, 27 Apr 2019 01:10:51 -0700 (PDT) X-Gm-Message-State: APjAAAUhWjW4scdQ2IFgseudaToCZiz/IPThqzS9TfhoHg+z3nlBOCr+ cVp5QAU64hsL3HkvVhKqSDh/YY1KBubDwWMd0is= X-Google-Smtp-Source: APXvYqxPVqk4xB8cImHFFD2xYj35o2M5yoUswLdDYwP21hCKUi3LyhN7GIMtekEAB+LIvB3HHjne2Icjsk1RmdChrQs= X-Received: by 2002:a37:a1c4:: with SMTP id k187mr28167955qke.341.1556352651161; Sat, 27 Apr 2019 01:10:51 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dmitriy Pavlov Date: Sat, 27 Apr 2019 11:10:45 +0300 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: TC build queue stall To: dev , Petr Ivanov Content-Type: multipart/alternative; boundary="000000000000eaf5a205877e96a0" --000000000000eaf5a205877e96a0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Ivan, thank you for the research! I think it may be related to agents' env. Petr, could you please take a look? Sincerely, =D1=81=D0=B1, 27 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 07:24, =D0=9F=D0= =B0=D0=B2=D0=BB=D1=83=D1=85=D0=B8=D0=BD =D0=98=D0=B2=D0=B0=D0=BD : > Also I cleaned up a build queue on TC a little bit removing builds > which cannot make any progress more than 10 hours. Please check your > recent builds. > > Most of them was ZooKeeper (Discovery) 3 [1]. It claims that there is > no compatible agents? Could not we run it on any Linux agent? > > Also it seems that there is a compatible agent trouble for a GCE [2] > build. Seems it need a special file on an agent > > Caused by: java.io.FileNotFoundException: > /home/teamcity/test_keys/gridgain-gce-key.p12 (No such file or > directory) > > [1] > https://ci.ignite.apache.org/viewType.html?buildTypeId=3DIgniteTests24Jav= a8_ZooKeeperDiscovery3&tab=3DcompatibilityList&branch_IgniteTests24Java8=3D= %3Cdefault%3E > [2] > https://ci.ignite.apache.org/viewType.html?buildTypeId=3DIgniteTests24Jav= a8_Gce&branch_IgniteTests24Java8=3D%3Cdefault%3E&tab=3DbuildTypeStatusDiv > > =D1=81=D0=B1, 27 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 07:21, =D0=9F=D0= =B0=D0=B2=D0=BB=D1=83=D1=85=D0=B8=D0=BD =D0=98=D0=B2=D0=B0=D0=BD : > > > > Igniters, > > > > Indeed it seems that an incident yesterday was caused by Russian > > characters in commit details. I think that following could be done: > > 1. File a bug against TC. > > 2. Do not use Russian characters in git user info, commit messages and > > other commit details. > > > > =D1=81=D0=B1, 27 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 07:20, =D0=9F= =D0=B0=D0=B2=D0=BB=D1=83=D1=85=D0=B8=D0=BD =D0=98=D0=B2=D0=B0=D0=BD : > > > > > > Igniters, > > > > > > Indeed it seems that an incident yesterday was caused by Russian > > > characters in commit details. I think that following could be done: > > > 1. File a bug against TC. > > > 2. Do not use Russian characters in git user info, commit messages an= d > > > other commit details. > > > > > > =D0=BF=D1=82, 26 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 10:19, =D0= =94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =D0=A1=D0=BE=D1=80=D0=BE=D0=BA=D0= =B8=D0=BD < > sbt.sorokin.dvl@gmail.com>: > > > > > > > > Hi Ivan! > > > > > > > > Thanks for investigation! > > > > I decided to try second your proposal first, and force pushed the > squashed commits just now. Let=E2=80=99s look whether it helped or not. > > > > > > > > =D0=BF=D1=82, 26 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 8:57, =D0= =9F=D0=B0=D0=B2=D0=BB=D1=83=D1=85=D0=B8=D0=BD =D0=98=D0=B2=D0=B0=D0=BD : > > > >> > > > >> I digged a little bit deeper into causes of the VCS problem report= ed > > > >> by TC. It looks like Russian characters in commit details (an auth= or > > > >> name here) drive TC crazy. Still not sure that it is the root caus= e > > > >> (quite weird if so). > > > >> > > > >> I found a PR with problematic commits [1]. Dmitrii Ryabov, Dmitriy > > > >> Sorokin and Mikhail Petrov commited to that PR. Guys could you > please > > > >> close the PR in order to check that it makes TC mad? It seems that > > > >> closing PR makes no harm because it can be easily reopened after. > Or, > > > >> for example squash commits and force push to PR avoiding Russian > > > >> characters. > > > >> > > > >> Need your assistance here. > > > >> > > > >> [1] https://github.com/apache/ignite/pull/6223/commits > > > >> > > > >> =D0=BF=D1=82, 26 =D0=B0=D0=BF=D1=80. 2019 =D0=B3. =D0=B2 06:10, = =D0=9F=D0=B0=D0=B2=D0=BB=D1=83=D1=85=D0=B8=D0=BD =D0=98=D0=B2=D0=B0=D0=BD <= vololo100@gmail.com>: > > > >> > > > > >> > Hi Igniters, > > > >> > > > > >> > Right now I am observing that many builds cannot make progress f= or > > > >> > many hours so far. TC shows that the majority of agents is free > [1]. > > > >> > But new builds seems fail to start. > > > >> > > > > >> > Does anyone know what is it and how to resolve it? Perhaps simpl= e > > > >> > reboot can help here. Also new codestyle check is a recent chang= e > and > > > >> > might be somehow related. > > > >> > > > > >> > Also I see problem with git checkout in some logs: > > > >> > [13:45:12] Collecting changes in 1 VCS root (7s) > > > >> > [13:45:12] [Collecting changes in 1 VCS root] VCS Root details > > > >> > [13:45:18] [Collecting changes in 1 VCS root] Compute revision f= or > > > >> > 'GitHub [apache/ignite]' > > > >> > [14:36:57] The build is removed from the queue to be prepared fo= r > the start > > > >> > [14:36:57] Starting the build on the agent aitc-lin12:06 > > > >> > [14:36:58] Clearing temporary directory: > /opt/buildagent/temp/buildTmp > > > >> > [14:36:58] Publishing internal artifacts > > > >> > [14:36:58] Clean build enabled: removing old files from > > > >> > /opt/buildagent/work/69588afcb2ab3382 > > > >> > [14:36:58] Checkout directory: > /opt/buildagent/work/69588afcb2ab3382 > > > >> > [14:36:58] Updating sources: server side checkout (running for > 15h:17m:37s) > > > >> > [14:36:58] [Updating sources] Building clean patch for VCS root: > > > >> > GitHub [apache/ignite] > > > >> > [14:39:20] [Updating sources] Failed to build patch for build > #11255 > > > >> > {build id=3D3698721, > buildTypeId=3DIgniteTests24Java8_CacheFailover2}, VCS > > > >> > root: "GitHub [apache/ignite]" {instance id=3D296, parent intern= al > > > >> > id=3D77, parent id=3DGitHubApacheIgnite, description: > > > >> > "https://github.com/apache/ignite.git#refs/heads/master"}, due t= o > > > >> > error: Patch building failed: > > > >> > org.apache.catalina.connector.ClientAbortException: > > > >> > java.net.SocketTimeoutException > > > >> > [14:39:20] [Updating sources] Transferring repository sources: > 36.93 > > > >> > MB so far... > > > >> > [02:40:02] [Updating sources] Transferring repository sources: > 36.95 > > > >> > MB so far... > > > >> > > > > >> > And a reported problem in "Overview" tab: > > > >> > Error collecting changes for VCS repository '"GitHub > [apache/ignite]" > > > >> > {instance id=3D296, parent internal id=3D77, parent > id=3DGitHubApacheIgnite, > > > >> > description: " > https://github.com/apache/ignite.git#refs/heads/master"}' > > > >> > > jetbrains.buildServer.serverSide.db.MySQL.MySqlIncorrectStringValueExcept= ion: > > > >> > Incorrect string value: '\xD0\xB4\xD0\xBC\xD0\xB8...' for column > > > >> > 'user_name' at row 1 while performing SQL query: SQL DML: insert > into > > > >> > vcs_history (modification_id, user_name, description, change_dat= e, > > > >> > vcs_root_id, version, display_version, changes_count, > register_date) > > > >> > values (?, ?, ?, ?, ?, ?, ?, ?, ?) | PARAMETERS: 882649, "=D0=B4= =D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 > > > >> > =D1=80=D1=8F=D0=B1=D0=BE=D0=B2 ", "fix wa= rming up throttle test\n", > > > >> > 1551457143000, 296, "06127b70b2060dade9bd652abc71f2b949508245", > > > >> > "06127b70b2060dade9bd652abc71f2b949508245", 1, 1556247859429: > > > >> > java.sql.SQLException: Incorrect string value: > > > >> > '\xD0\xB4\xD0\xBC\xD0\xB8...' for column 'user_name' at row 1 > > > >> > > > > >> > [1] > https://ci.ignite.apache.org/viewType.html?buildTypeId=3DIgniteTests24Jav= a8_RunAll&tab=3DbuildTypeBranches > > > >> > > > > >> > -- > > > >> > Best regards, > > > >> > Ivan Pavlukhin > > > >> > > > >> > > > >> > > > >> -- > > > >> Best regards, > > > >> Ivan Pavlukhin > > > > > > > > > > > > -- > > > Best regards, > > > Ivan Pavlukhin > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > > > -- > Best regards, > Ivan Pavlukhin > --000000000000eaf5a205877e96a0--