From dev-return-112114-archive-asf-public=cust-asf.ponee.io@cloudstack.apache.org Fri Nov 9 16:16:56 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id E6491180627 for ; Fri, 9 Nov 2018 16:16:55 +0100 (CET) Received: (qmail 481 invoked by uid 500); 9 Nov 2018 15:16:54 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 469 invoked by uid 99); 9 Nov 2018 15:16:54 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Nov 2018 15:16:54 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id C0F9DC2BC1 for ; Fri, 9 Nov 2018 15:16:53 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.859 X-Spam-Level: * X-Spam-Status: No, score=1.859 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.14, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=bw-sw-com.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id LAswmOATiO50 for ; Fri, 9 Nov 2018 15:16:50 +0000 (UTC) Received: from mail-vs1-f51.google.com (mail-vs1-f51.google.com [209.85.217.51]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id BCD62623F6 for ; Fri, 9 Nov 2018 15:16:49 +0000 (UTC) Received: by mail-vs1-f51.google.com with SMTP id p74so1232702vsc.0 for ; Fri, 09 Nov 2018 07:16:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bw-sw-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=z+JpzbEKeFYPitouioFNvfk5KMAKjzSvb5YcsxCNfZ8=; b=fz6m+Spumaelz6fxO8PuyyhJl8NgW6IUStywMCtiracaX6HmHmGWUV5hOqvMt688Zk l4izRY3K+7Na7dxfh+x5GFy2ubXRuWhMGdYy5y545q/a3mCIkx+teJiZPAjExU8uNhZL 7d+nsfpghQJuY+FTPMz9HSW64izT6XaN4HC62MaYaNuyTjPRlbG+nbgTaZQKuMV134OV ObfRcCM+nIk/y9FOzC2uCtKl9cbFy2+HTklj7w+3X5yJPeOJq2Y5HVMb9IBN9fXuQ7mN Kik0vzN/w4bLA8tlXMDtuux+ERUmh1ENNHLLVwmOgFvW8oyI652Nh5kMlv6jwoWHE19y ocwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=z+JpzbEKeFYPitouioFNvfk5KMAKjzSvb5YcsxCNfZ8=; b=MqIRniCaXNIGkBkO3ZvTP0LQBx9cM7qCxhtleSxbVmBHUXhR2io28/mXsZxWbfNhPP m1swrvU93Zicxbb5laooPBR2jjS8yNfGH9VWUxa0l7fTHwatvw9xA/GkaFuBX4SqGM6b zeTipCPc0cRJHafooHUDfAIZK8CH6a9HyUDUdFP9X29WjtmwExFvdj4VjGt3iRYhpv0/ e5Og+WdeXyXxgqXDn106WcFsooVy3N2OFsO6rJRsG+m8Oh2GnMbbW4r9VEKsf/A7MdJ3 S7gtFm5ag9M+F2DXiasDRwEf+WtRtIwoVr4azf4ZiFB/UGSemKeiB9T50KCofGcMJ3ns fUyA== X-Gm-Message-State: AGRZ1gJmp6HriZOmCf7peTpwz+lySKhJfAO8Yw1VK6j8gxWoJ2eAW3b9 IKhMLZ5Rdh3onDsgZrUy9grrbbMUnrHN0fIEt3b1KYiu17Q= X-Google-Smtp-Source: AJdET5dL6VozqSw/Cr8BuG9bA5kIyqIOxf8f842pJtq2WHZldZltduHJDDkHEWcKLGwQKgONZWmytzR4LT6LNrN4GGA= X-Received: by 2002:a67:4d4f:: with SMTP id a76mr3579357vsb.167.1541776608512; Fri, 09 Nov 2018 07:16:48 -0800 (PST) MIME-Version: 1.0 References: <40DA8B05-5C27-4696-BBEF-CD8983C42C8D@shapeblue.com> <5EF5B369-99D1-4DE3-BA18-39A4AFBD58EF@shapeblue.com> In-Reply-To: <5EF5B369-99D1-4DE3-BA18-39A4AFBD58EF@shapeblue.com> From: Ivan Kudryavtsev Date: Fri, 9 Nov 2018 10:16:37 -0500 Message-ID: Subject: Re: RFC. Marvin tests fail To: dev Content-Type: multipart/alternative; boundary="00000000000012e68f057a3cd783" --00000000000012e68f057a3cd783 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello Boris, During the troubleshooting, I have found that the problem is connected with my host environment, so I decided to build the Dockerfile which is for "simulator & client" running altogether. The Dockerfile itself is, for example, is here: https://pastebin.com/raw/ZVjtHX7F The Dockerfile is redesigned from the original simulator the way to be fast built for QA E2E tests from the point of view: - java code changes very rarely; - marvin code changes rarely; - tests change often. After being built it can be used like: Simulator run: docker run --name sim -v my-tmp:/tmp -it --rm simulator:4.11 Tests run: docker exec -it sim bash /root/docker_run_tests.sh If the community is interested in then I can supersede original tools/docker/Dockerfile with that one. Also, I added .dockerignore to avoid Docker cache invalidation. So, to sum up, this Dockerfile is for easy Marvin development and testing in the isolated, stable environment. Give me your feedback, please. All changes are here: https://github.com/apache/cloudstack/pull/3012/files =D0=BF=D1=82, 9 =D0=BD=D0=BE=D1=8F=D0=B1. 2018 =D0=B3. =D0=B2 2:07, Boris S= toyanov : > you could run it with a debugger in PyCharm for example and figure out if > there=E2=80=99s anything wrong with the code. Please note that these test= s are > successful with KVM, VMware and Xen. > > The logs you=E2=80=99ve shared are just results, there=E2=80=99s a failed= _plus_exceptions > file in each test file log, where you could find more detailed info and > stack trace. > > Boris. > > > boris.stoyanov@shapeblue.com > www.shapeblue.com > Amadeus House, Floral Street, London WC2E 9DPUK > @shapeblue > > > > > On 8 Nov 2018, at 2:37, Ivan Kudryavtsev > wrote: > > > > Looks like the reason for the failure is in python code, not tests. > Someone > > broke tests. > > > > =D1=81=D1=80, 7 =D0=BD=D0=BE=D1=8F=D0=B1. 2018 =D0=B3. =D0=B2 19:29, Iv= an Kudryavtsev >: > > > >> Well, when I run outside of docker. I see another behavior: > >> Those OOB tests work well, but I meet errors in other pieces of code: > >> > >> *ivan@notebook:/tmp/MarvinLogs/Nov_07_2018_18_53_35_8SWDQD$ grep -P > >> 'EXCEPTION|FAIL' results.txt * > >> Tests ha enable/disable feature at cluster and zone level ... =3D=3D= =3D > >> TestName: test_ha_configure_enabledisable_across_clusterzones | Status= : > >> EXCEPTION =3D=3D=3D > >> Tests ha resource ownership expiry across multi-mgmt server ... =3D=3D= =3D > >> TestName: test_ha_multiple_mgmt_server_ownership | Status : EXCEPTION > =3D=3D=3D > >> Tests ha FSM transitions for valid healthy host ... =3D=3D=3D TestName= : > >> test_ha_verify_fsm_available | Status : EXCEPTION =3D=3D=3D > >> Tests ha FSM transitions leading to degraded state ... =3D=3D=3D TestN= ame: > >> test_ha_verify_fsm_degraded | Status : EXCEPTION =3D=3D=3D > >> Tests ha FSM transitions for failures leading to fenced state ... =3D= =3D=3D > >> TestName: test_ha_verify_fsm_fenced | Status : EXCEPTION =3D=3D=3D > >> Tests ha FSM transitions leading to recovering ... =3D=3D=3D TestName: > >> test_ha_verify_fsm_recovering | Status : EXCEPTION =3D=3D=3D > >> =3D=3D=3D TestName: test_list_zones_metrics | Status : EXCEPTION =3D= =3D=3D > >> Tests out-of-band management ownership expiry across multi-mgmt server > ... > >> =3D=3D=3D TestName: test_oobm_multiple_mgmt_server_ownership | Status = : > FAILED =3D=3D=3D > >> FAIL > >> =3D=3D=3D TestName: test_ha_configure_enabledisable_across_clusterzone= s | > Status > >> : EXCEPTION =3D=3D=3D > >> =3D=3D=3D TestName: test_ha_multiple_mgmt_server_ownership | Status : > EXCEPTION > >> =3D=3D=3D > >> =3D=3D=3D TestName: test_ha_verify_fsm_available | Status : EXCEPTION = =3D=3D=3D > >> =3D=3D=3D TestName: test_ha_verify_fsm_degraded | Status : EXCEPTION = =3D=3D=3D > >> =3D=3D=3D TestName: test_ha_verify_fsm_fenced | Status : EXCEPTION =3D= =3D=3D > >> =3D=3D=3D TestName: test_ha_verify_fsm_recovering | Status : EXCEPTION= =3D=3D=3D > >> FAIL: Tests out-of-band management ownership expiry across multi-mgmt > >> server > >> =3D=3D=3D TestName: test_oobm_multiple_mgmt_server_ownership | Status = : FAILED > >> =3D=3D=3D > >> FAILED (SKIP=3D13, errors=3D6, failures=3D1) > >> > >> I pasted errors to Pastebin: https://pastebin.com/tXw6mRk7 > >> > >> > >> > >> > >> =D1=81=D1=80, 7 =D0=BD=D0=BE=D1=8F=D0=B1. 2018 =D0=B3. =D0=B2 14:05, B= oris Stoyanov < > boris.stoyanov@shapeblue.com > >>> : > >> > >>> Hi Ivan, > >>> I guess you=E2=80=99re referring to Out of Band management? I think t= here was a > >>> simulator provider for those tests and they are reported as passing > with > >>> latest KVM test run. > >>> > >>> Can you share any logs/exceptions? I don=E2=80=99t see anything wrong= with your > >>> setup on a first glance btw. > >>> > >>> Bobby. > >>> > >>> > >>> boris.stoyanov@shapeblue.com > >>> www.shapeblue.com > >>> Amadeus House, Floral Street, London WC2E 9DPUK > >>> @shapeblue > >>> > >>> > >>> > >>>> On 7 Nov 2018, at 20:52, Ivan Kudryavtsev > >>> wrote: > >>>> > >>>> Hello, dev team. Now I try to put a PR and need to write marvin test > for > >>>> it. I did it in the past, so tried to recall how to run the ecosyste= m. > >>>> > >>>> I started from the document: > >>>> > >>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+w= ith+Python > >>>> > >>>> To run simulator I have built docker image for 4.11: > >>>> > >>>> docker build -t simulator:4.11 -f tools/docker/Dockerfile . > >>>> > >>>> It has built correctly. > >>>> > >>>> Next, I run the simulator: > >>>> docker run -it --rm -p 8080:8080 -p 8096:8096 simulator:411 > >>>> > >>>> It started well, I can login. > >>>> > >>>> Next, I deployed DC as specified in document above: > >>>> python tools/marvin/marvin/deployDataCenter.py > >>>> < > >>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/deployDataCenter.p= y > >>>> > >>>> \ > >>>> -i setup/dev/advanced.cfg > >>>> > >>>> > >>>> Next, I run the suite: > >>>> $ export MARVIN_CONFIG=3Dsetup/dev/advanced.cfg > >>>> > >>>> $ export TEST_SUITE=3Dtest/integration/smoke > >>>> $ export ZONE_NAME=3DSandbox-simulator > >>>> $ nosetests-2.7 \ > >>>> --with-marvin \ > >>>> --marvin-config=3D${MARVIN_CONFIG} \ > >>>> -w ${TEST_SUITE} \ > >>>> --with-xunit \ > >>>> --xunit-file=3D/tmp/bvt_selfservice_cases.xml > >>>> < > >>> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/bvt_selfservice_ca= ses.xml > >>>> > >>>> \ > >>>> --zone=3D${ZONE_NAME} \ > >>>> --hypervisor=3Dsimulator \ > >>>> -a tags=3Dadvanced,required_hardware=3Dfalse > >>>> > >>>> What I see is that some tests fail, especially those, which connecte= d > >>> with > >>>> OOB management. > >>>> I tried to do the same for basic zone with basic.cfg and also OOB > tests > >>>> fail and some other as well. > >>>> > >>>> Is it OK or I do something wrong? > >>>> > >>>> Thanks a lot. > >>>> > >>>> > >>>> -- > >>>> With best regards, Ivan Kudryavtsev > >>>> Bitworks LLC > >>>> Cell RU: +7-923-414-1515 > >>>> Cell USA: +1-201-257-1512 > >>>> WWW: http://bitworks.software/ > >>> > >>> > >> > >> -- > >> With best regards, Ivan Kudryavtsev > >> Bitworks LLC > >> Cell RU: +7-923-414-1515 > >> Cell USA: +1-201-257-1512 > >> WWW: http://bitworks.software/ > >> > >> > > > > -- > > With best regards, Ivan Kudryavtsev > > Bitworks LLC > > Cell RU: +7-923-414-1515 > > Cell USA: +1-201-257-1512 > > WWW: http://bitworks.software/ > > --=20 With best regards, Ivan Kudryavtsev Bitworks LLC Cell RU: +7-923-414-1515 Cell USA: +1-201-257-1512 WWW: http://bitworks.software/ --00000000000012e68f057a3cd783--