From dev-return-31691-archive-asf-public=cust-asf.ponee.io@geode.apache.org Thu Aug 22 16:35:54 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id B533E180644 for ; Thu, 22 Aug 2019 18:35:53 +0200 (CEST) Received: (qmail 36708 invoked by uid 500); 22 Aug 2019 16:35:53 -0000 Mailing-List: contact dev-help@geode.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@geode.apache.org Delivered-To: mailing list dev@geode.apache.org Received: (qmail 36697 invoked by uid 99); 22 Aug 2019 16:35:53 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Aug 2019 16:35:53 +0000 Received: from mail-ot1-f69.google.com (mail-ot1-f69.google.com [209.85.210.69]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id DEB505896 for ; Thu, 22 Aug 2019 16:35:52 +0000 (UTC) Received: by mail-ot1-f69.google.com with SMTP id c1so3389231otb.22 for ; Thu, 22 Aug 2019 09:35:52 -0700 (PDT) X-Gm-Message-State: APjAAAU3P5JyEtyH9SBhmUA8hxfrIp9RF1ZGP8+l5AL/4GqHZhda/g3W kwdtK5s//aoCRb9ClpE149iav11VlGa4qaQVASPRJFrLSnFamiK0vCpVPfyjkSpSV8HfGvm1Gxo cFbEYS9TK/WJtt7epY89jOvZoge7k/TiCpbvGljLiuXJIXJ3C0Rs+aMY= X-Received: by 2002:aca:750c:: with SMTP id q12mr6189oic.177.1566491752112; Thu, 22 Aug 2019 09:35:52 -0700 (PDT) X-Google-Smtp-Source: APXvYqw3ePU4Sc3ZGDckohtpFAcdHyetdrgITBSaa7LJRYS6mfyN2xgL5+P1uhZFJ16rAER+s0f7GH8gx8D37iE1d/U= X-Received: by 2002:aca:750c:: with SMTP id q12mr6180oic.177.1566491751905; Thu, 22 Aug 2019 09:35:51 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Kirk Lund Date: Thu, 22 Aug 2019 09:35:40 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Need help: Jetty dunit tests blocking creation of geode-log4j To: geode Content-Type: multipart/alternative; boundary="0000000000006aba2d0590b74897" --0000000000006aba2d0590b74897 Content-Type: text/plain; charset="UTF-8" So far Jens and I are unable to reproduce these test failures [1]. This does not appear to be a test or java code problem. Based on google searches [2] [3], the cause might be either: 1) the distributed test job is out of swap space when the Tomcat/Jetty tests run 2) the /proc/sys/vm/overcommit_memory and /proc/sys/vm/overcommit_ratio may be misconfigured I don't have sufficient concourse knowledge to attempt to fix or change either of the above two settings in our precheckin environment. Can someone please help me with this? The failure: *OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f74a4ba4000, 65536, 1) failed; error='Not enough space' (errno=12) [thread 26510 also had an error] * [1] https://concourse.apachegeode-ci.info/builds/88240 [2] https://bugs.openjdk.java.net/browse/JDK-8216619 [3] https://stackoverflow.com/questions/46801741/jvm-crashes-with-error-cannot-allocate-memory-errno-12 On Wed, Aug 21, 2019 at 8:36 AM Jens Deppe wrote: > I can work with you on this if you're still blocked. > > --Jens > > On Tue, Aug 20, 2019 at 4:12 PM Kirk Lund wrote: > > > Does anyone know how to debug geode-assembly Jetty dunit tests that fail > to > > launch modify_war? > > > > It passes 100% locally in intellij and with gradle cli. It only fails in > > concourse PR precheckin. > > > > Right now, this is the last thing blocking me from submitting a non-draft > > PR to move all log4j-core code from geode-core to geode-log4j. This is > > blocking the creation of geode-log4j. > > > > The only changes in my branch are moving all log4j-core code from > > geode-core to geode-log4j. > > > > If anyone else wants to see this change make it to develop, then I need > > help! > > > > When the test tries to execute modify_war, it fails with the following > > output and stack trace. No other info is available for debugging as > > apparently this kills the gradle daemon. > > > > Failed PR precheckin dunit job: > > https://concourse.apachegeode-ci.info/builds/88240 > > PR: https://github.com/apache/geode/pull/3914 > > > > > Task :geode-assembly:distributedTest > > > > org.apache.geode.session.tests.Jetty9CachingClientServerTest > > > containersShouldHavePersistentSessionData FAILED > > java.lang.RuntimeException: Something very bad happened when trying > to > > start container > > > > > JETTY9_client-server_containersShouldHavePersistentSessionData_0_a6ebd229-072b-47db-a9bf-ca3713175f05_ > > > > Caused by: > > java.lang.RuntimeException: Something very bad happened to this > > container when starting. Check the cargo_logs folder for container logs. > > > > Caused by: > > java.io.IOException: Unable to run modify_war script, > command: > > > > > [/tmp/geode_container_install17845041006471328987/cargo_modules/Apache_Geode_Modules-1.11.0-SNAPSHOT-AppServer/bin/modify_war, > > -J, -Xmx2096m, -w, > > > > > /home/geode/geode/geode-assembly/build/distributedTest254/../../../extensions/session-testing-war/build/libs/session-testing-war.war, > > -t, client-server, -o, > > > > > /tmp/geode_container_install17845041006471328987/cargo_wars/JETTY9_client-server_containersShouldHavePersistentSessionData_0_a6ebd229-072b-47db-a9bf-ca3713175f053692095078744488223.war, > > -p, gemfire.cache.enable_local_cache=true, -p, > > > > > gemfire.property.log-file=/home/geode/geode/geode-assembly/build/distributedTest254/cargo_logs/JETTY9_client-server_containersShouldHavePersistentSessionData_0_a6ebd229-072b-47db-a9bf-ca3713175f05/gemfire.log, > > -p, > > > > > gemfire.property.cache-xml-file=/home/geode/geode/geode-assembly/build/distributedTest254/cargo_logs/JETTY9_client-server_containersShouldHavePersistentSessionData_0_a6ebd229-072b-47db-a9bf-ca3713175f05/cache-client.xml] > > log file: > > ERROR: Error updating web.xml > > ng: INFO: os::commit_memory(0x000000077d000000, 2147483648, > 0) > > failed; error='Not enough space' (errno=12) > > > --0000000000006aba2d0590b74897--