From dev-return-1864-archive-asf-public=cust-asf.ponee.io@mxnet.incubator.apache.org Fri Jan 12 01:49:08 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 901CC18076D for ; Fri, 12 Jan 2018 01:49:08 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 80B04160C13; Fri, 12 Jan 2018 00:49:08 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id A942E160C41 for ; Fri, 12 Jan 2018 01:49:07 +0100 (CET) Received: (qmail 26567 invoked by uid 500); 12 Jan 2018 00:49:03 -0000 Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@mxnet.incubator.apache.org Delivered-To: mailing list dev@mxnet.incubator.apache.org Received: (qmail 26144 invoked by uid 99); 12 Jan 2018 00:49:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 Jan 2018 00:49:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D998DC1C18 for ; Fri, 12 Jan 2018 00:49:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.129 X-Spam-Level: ** X-Spam-Status: No, score=2.129 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id xHbxsBprCRbN for ; Fri, 12 Jan 2018 00:49:01 +0000 (UTC) Received: from mail-it0-f52.google.com (mail-it0-f52.google.com [209.85.214.52]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 4A2695F3CC for ; Fri, 12 Jan 2018 00:49:01 +0000 (UTC) Received: by mail-it0-f52.google.com with SMTP id d137so7315199itc.2 for ; Thu, 11 Jan 2018 16:49:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=GElLggzUug/uIEni6CeP4wV9Qtd0yI7Cs9EBxkU5wEk=; b=APWcnFB4+nQVS8PYC8Choc0r0G5DdOTwixfjWGF7RbZHr5cCaaBAQ8vvkS8HJIbT/p fiUUXVq5F8FhxeVIgtlWQCH+0i1tEc3iaT457zSzeSH+eoWUdizDeNu1bCg+DlN2nLMe IZ5oS6LOVQAiP1yiG9CoVzYM0z7yyv8w0JCwe7FIbJuAj/18jy5bXQeet4/j0PnOZ6NS /kM6EBzkCl0F3qEInrxTBkUH2iVTIIRaXd3qNofegWj2T5Wij1f3m61twxT9t6Fzm1sz Su3FTz4IXvRqS1I4cqmVVwx8TdGoZgNNw9/oa4cQDP8S0a4P9defgoJH0lgal1NYrlkz qxLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=GElLggzUug/uIEni6CeP4wV9Qtd0yI7Cs9EBxkU5wEk=; b=W9Jqz3rzzA4vZZ6iAcFRC0UV3Hb8bfnPbJjgyglhO1hiRaF14+P4p9mLQnfT0v73/t JAb0MnmqqBWohPF2JSdVn3ViOxWuC8VcC9iPT1RUWgK4RDA219/IsLuf7yA/76zPiZO8 7E600U1Mrra6J0D4EL35I6otgxXyLRIxTKsuKW3GFO1F3o+4UjmH7hIkgsfr10eWAGIR B7iBAt0BfOu7q3CA8rP7BrEvHEr5z4VwDz3CiovpHUPRcEWoJX2US+iAhWjIbx+uEpBt BJjCpHV8CcqApz3HCDVjYpvcCrUf1okEDCGUt8AdQc1AQh0xT5PUlrDdo2s6CwxkIEHr D52g== X-Gm-Message-State: AKwxyte752xyOtryg52hZbd5D1zm5WPTaYLNFMLghhBg4wuEFGwLqFOe 0lxtgaTEm8MrJXECBx2IGfx7FzmxSgoNQ8Mqqhw= X-Google-Smtp-Source: ACJfBotnIjH+yNJUEgUeh5Dw6L8cOLF99aLY90S76OQpaw3jjwlUH3YD285cBYjSMjEbRtLUTs+YfkDVoMM+CFUslio= X-Received: by 10.36.125.205 with SMTP id b196mr3264082itc.128.1515718140637; Thu, 11 Jan 2018 16:49:00 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.164.79 with HTTP; Thu, 11 Jan 2018 16:49:00 -0800 (PST) In-Reply-To: References: From: Chris Olivier Date: Thu, 11 Jan 2018 16:49:00 -0800 Message-ID: Subject: Re: CI failure due to offline llvm.org To: dev@mxnet.incubator.apache.org Content-Type: multipart/alternative; boundary="001a114452005a486e056289a1f4" --001a114452005a486e056289a1f4 Content-Type: text/plain; charset="UTF-8" what is deleteDir() call doing in Jenkinsfile? Yes, I mentioned the case where it wasn't getting cleaned. On Thu, Jan 11, 2018 at 4:41 PM, Marco de Abreu < marco.g.abreu@googlemail.com> wrote: > During git_init: First we're just using git clean, if checkout fails, we're > deleting the entire workspace and retrying. > > During build: First we're using regular make. If build fails, we're using > make clean before executing make again. > > During test: No cleanup happening in case of failure. > > So far, I haven't noticed any files not being deleted in the workspace. Do > you know an example? > > -Marco > > On Fri, Jan 12, 2018 at 1:34 AM, Chris Olivier > wrote: > > > What approach is used now? I see in Jenkinsfile() that deleteDir() is > > called at the top of init_git() and init_git_win(). That dele5tes the > > whole directory, correct? > > > > Before there were problems with 'git clean -d -f' *not* deleting some > > directories which were tracked on one branch and not on another, which I > > believe is why deletDir() was put there. The directory I recall was > > something like lua-package or something that was in someone's private > repo > > or something like that... > > > > On Thu, Jan 11, 2018 at 4:02 PM, Marco de Abreu < > > marco.g.abreu@googlemail.com> wrote: > > > > > While it's a quite harsh solution to delete the entire workspace, I > think > > > that it's a good way. Git checkout takes between 2 and 10 seconds, so I > > > don't think we need to optimize in that regard. > > > > > > git clean is our 'soft' approach to clean up. Deleting the workspace is > > the > > > 'hard' approach, so this shouldn't be an issue. > > > > > > But there is one catch: Windows builds are not containerized and while > we > > > delete the workspace, there could still be a lot of files which are not > > > being tracked. In future I'd like to have at least a file-system-layer > in > > > between our tests and the host, but we will have to analyze if > something > > > like this exists. At the moment, we even got tests writing to system32. > > > > > > -Marco > > > > > > On Fri, Jan 12, 2018 at 12:44 AM, Chris Olivier > > > > wrote: > > > > > > > Ok, but still on that note. I remember before that when some problems > > > were > > > > being fixed in CI (before your time), they switched to deleting the > > > entire > > > > source directory, ".git" subdirectory and all. At the time, the CI > was > > > in > > > > such an chaotic state that I didn't make an issue of it, but now that > > it > > > > has stabilized (for the most part, today's incident > notwithstanding), I > > > > think that we may want to revisit it if it is still doing that. you > > > could, > > > > for example, just delete everything except the .git directory and > then > > > do a > > > > 'git reset --hard' to get back a baseline before having to > re-download > > > > everything every tim e(also should speed up the builds). > > > > > > > > Note that 'git clean' was not working as it doesn't delete 'unknown' > > > > directories, which was the problem. > > > > > > > > WDYT? > > > > > > > > On Thu, Jan 11, 2018 at 3:26 PM, Marco de Abreu < > > > > marco.g.abreu@googlemail.com> wrote: > > > > > > > > > This happens because we just merged the clang compilation > > > > > https://github.com/apache/incubator-mxnet/commit/ > > > > > 2b73aac527a3439ec0dc9b1e76c6df09ea347eb1. > > > > > This means that clang has to get installed on all slaves and after > > some > > > > > time, the docker images will be cached. The problem right now is > that > > > > their > > > > > apt-server is unavailable, means the initial installation to create > > the > > > > > docker cache doesn't succeed. In future, this will be cached. > > > > > > > > > > -Marco > > > > > > > > > > On Thu, Jan 11, 2018 at 11:45 PM, Chris Olivier < > > cjolivier01@gmail.com > > > > > > > > > wrote: > > > > > > > > > > > do we download all submodules from scratch every build? if we > do > > > then > > > > > we > > > > > > should probably find a way not to suggest just doing git reset or > > > > > something > > > > > > like that > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jan 11, 2018 at 1:47 PM Marco de Abreu < > > > > > > marco.g.abreu@googlemail.com> > > > > > > wrote: > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > we're currently experiencing a CI outage caused by > > > > http://apt.llvm.org > > > > > > not > > > > > > > being reachable. > > > > > > > > > > > > > > Best regards, > > > > > > > Marco > > > > > > > > > > > > > > > > > > > > > > > > > > > > --001a114452005a486e056289a1f4--