From users-return-33280-archive-asf-public=cust-asf.ponee.io@cloudstack.apache.org Thu Jul 25 14:48:50 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 0435E18066C for ; Thu, 25 Jul 2019 16:48:49 +0200 (CEST) Received: (qmail 89784 invoked by uid 500); 25 Jul 2019 14:48:48 -0000 Mailing-List: contact users-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@cloudstack.apache.org Delivered-To: mailing list users@cloudstack.apache.org Received: (qmail 89766 invoked by uid 99); 25 Jul 2019 14:48:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Jul 2019 14:48:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 4F8091A33ED for ; Thu, 25 Jul 2019 14:48:47 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.004 X-Spam-Level: ** X-Spam-Status: No, score=2.004 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, KAM_SHORT=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=emzior-ru.20150623.gappssmtp.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id GYfRPbaxkfne for ; Thu, 25 Jul 2019 14:48:45 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::244; helo=mail-lj1-x244.google.com; envelope-from=d.berezhnoy@emzior.ru; receiver= Received: from mail-lj1-x244.google.com (mail-lj1-x244.google.com [IPv6:2a00:1450:4864:20::244]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 7EC077DC04 for ; Thu, 25 Jul 2019 14:48:44 +0000 (UTC) Received: by mail-lj1-x244.google.com with SMTP id y17so23719047ljk.10 for ; Thu, 25 Jul 2019 07:48:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=emzior-ru.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=52YR7Su3LRaSvh4HtlUCG3yhcci8BpLRS8P+kCp5CSI=; b=fF3ouMlRkpLC+jQza0wgAf+HKyyzLTzrdQhZpkL7mcZcLmP7WxYDqvD/5hq8wX+0Pe f4Ywkunif7jcJa4ZL6s2EK1alsm44YMUO/UM1PE17wHitU9gaj3JuG7pE8taexenboXH ETlbxRwggt5q8hYjDOkMUS+J59YWmQV43xOcryKacNQFNIHYfemAjwZYe3w5ZGwro+VD q8bSrFCqpAa9GoGck9DGEFHlHMtWRAlOHxX0v/rTqUU+G/QG2i+RDTuzHt+YAeI4f9lf uvBYcum4oJIxFp/FjSGrrEwf4c5YJNLdd0R6nV9Z+YkSEwXKLQ/VH/uo92bK+mgAymzU S8XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=52YR7Su3LRaSvh4HtlUCG3yhcci8BpLRS8P+kCp5CSI=; b=GWfAHD3XU1KqXRMteuzJYgXWjdJZ4liHiv8Ha5qWIUCC5Ai2NLKeC459/gewc2qRmz YTZp+wseup5JAzJY+yYPgX6k0QVBcMlefyQEipTHYrqxF5apWP3UmRjPpomi2j6cxBJI 44q9OCNb8O3re6YSc/KvJboS7SbOwp48U2Y++fqGEFlfizlQSgYCP6op6JA9V0uYgnuj eZLm9zTGfaRwJZa7iMSJU86t/KRPkOmuj9RTkZNdNVH38rvuyymzgX2oea5dv+oTrD1Z 9sf2hbfBPkLRhw1cFYO0pK53TpdQRCd6Q1XaVCOVZiWcIpuLn4pvHUldINg7LEIkaOM3 zOqA== X-Gm-Message-State: APjAAAVtCg8VIBuY7GbVC173COzg8Pq79A1MqZrpwQC+c71B4+hk36RL Fse6aYE+xNZrJy33Mwrlgs5yQq0vfRRSGzO3XhYvdRcZ X-Google-Smtp-Source: APXvYqyuRYe0EZW3YajDh5zPU4eLHw/4ra+EeECn7WgASNpCHl7olbyZq7b57d+AvIi3Mx7FDXmUPsu2Fu98RUgck20= X-Received: by 2002:a2e:900c:: with SMTP id h12mr44286469ljg.197.1564066123526; Thu, 25 Jul 2019 07:48:43 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dmitry Berezhnoy Date: Thu, 25 Jul 2019 17:48:59 +0300 Message-ID: Subject: Re: Failed to get free memory without no reason To: users@cloudstack.apache.org Content-Type: multipart/alternative; boundary="000000000000b2c61f058e828510" --000000000000b2c61f058e828510 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Log from that CloudStack station really huge, so I download log from another CloudStack(version 4.10.0) with the same errors. logid:f0c4abda for example. 2019-07-25 15:57:58,895 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-149:ctx-d621076d job-6654658 ctx-d5e8eff7 FirstFitRoutingAllocator) (logid:f0c4abda) Free RAM: 88188272640 , Requested RAM: 402653184 2019-07-25 15:57:58,895 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-149:ctx-d621076d job-6654658 ctx-d5e8eff7 FirstFitRoutingAllocator) (logid:f0c4abda) Host has enough CPU and RAM available 2019-07-25 15:57:58,895 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-149:ctx-d621076d job-6654658 ctx-d5e8eff7 FirstFitRoutingAllocator) (logid:f0c4abda) STATS: Can alloc CPU from host: 1, used: 48556, reserved: 0, actual total: 144000, total with overprovisioning: 144000; requested cpu:384,alloc_from_last_host?:false ,considerReservedCapacity?: true 2019-07-25 15:57:58,895 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-149:ctx-d621076d job-6654658 ctx-d5e8eff7 FirstFitRoutingAllocator) (logid:f0c4abda) STATS: Can alloc MEM from host: 1, used: 45902462976, reserved: 0, total: 134090735616; requested mem: 402653184,alloc_from_last_host?:false ,considerReservedCapacity?: true ... 2019-07-25 15:58:07,660 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-27:ctx-bce5a678 job-6654658/job-6654659 ctx-fe95df77) (logid:f0c4abda) Unable to start VM on Host[-1-Routing] due to failed to get free memory ... 2019-07-25 15:58:14,421 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-27:ctx-bce5a678 job-6654658/job-6654659 ctx-fe95df77) (logid:f0c4abda) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-977270-VM]Scope=3Dinterface com.cloud.dc.DataCenter; id=3D3 management-server.log https://yadi.sk/d/wzyZVeTKS2kkDQ =D1=87=D1=82, 25 =D0=B8=D1=8E=D0=BB. 2019 =D0=B3. =D0=B2 16:51, Andrija Pan= ic : > Can you share the whole log file (pastebin.org please, or similar) ? > > On Wed, 24 Jul 2019 at 17:34, Dmitry Berezhnoy > wrote: > > > Hello, > > > > Asynchronous creating VM bring to "Unable to start VM on Host[-1-Routin= g] > > due to failed to get free memory". After that I see ERROR: > > InsufficientServerCapacityException. > > In previous messages I see large amount of free resources: > > 2019-07-24 04:50:45,443 DEBUG [c.c.c.CapacityManagerImpl] > > (Work-Job-Executor-82:ctx-8d1155a3 job-2281443/job-2281455 ctx-52b48ca6= ) > > (logid:b6e33d80) Current Used CPU: 115708 , Free CPU:76356 ,Requested > CPU: > > 512 > > 2019-07-24 04:50:45,443 DEBUG [c.c.c.CapacityManagerImpl] > > (Work-Job-Executor-82:ctx-8d1155a3 job-2281443/job-2281455 ctx-52b48ca6= ) > > (logid:b6e33d80) Current Used RAM: 122003914752 , Free RAM:145667670016 > > ,Requested RAM: 536870912 > > 2019-07-24 04:50:45,443 DEBUG [c.c.c.CapacityManagerImpl] > > (Work-Job-Executor-82:ctx-8d1155a3 job-2281443/job-2281455 ctx-52b48ca6= ) > > (logid:b6e33d80) CPU STATS after allocation: for host: 1, old used: > 115708, > > old reserved: 1536, actual total: 193600, total with overprovisioning: > > 193600; new used:116220, reserved:1536; requested > > cpu:512,alloc_from_last:false > > 2019-07-24 04:50:45,443 DEBUG [c.c.c.CapacityManagerImpl] > > (Work-Job-Executor-82:ctx-8d1155a3 job-2281443/job-2281455 ctx-52b48ca6= ) > > (logid:b6e33d80) RAM STATS after allocation: for host: 1, old used: > > 122003914752, old reserved: 1610612736, total: 269282197504; new used: > > 122540785664, reserved: 1610612736; requested mem: > > 536870912,alloc_from_last:false > > > > How it possible? > > > > Thanks in advance for the help, > > Dmitry. > > > > > -- > > Andrija Pani=C4=87 > --000000000000b2c61f058e828510--