From dev-return-40896-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Wed Oct 24 10:26:05 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 11C38180638 for ; Wed, 24 Oct 2018 10:26:04 +0200 (CEST) Received: (qmail 97870 invoked by uid 500); 24 Oct 2018 08:26:04 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 97858 invoked by uid 99); 24 Oct 2018 08:26:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Oct 2018 08:26:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id CE998C934D for ; Wed, 24 Oct 2018 08:26:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.211 X-Spam-Level: *** X-Spam-Status: No, score=3.211 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URI_HEX=1.313] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id em1d_Q178VQs for ; Wed, 24 Oct 2018 08:26:00 +0000 (UTC) Received: from mail-vs1-f42.google.com (mail-vs1-f42.google.com [209.85.217.42]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id B32FA5F3AD for ; Wed, 24 Oct 2018 08:26:00 +0000 (UTC) Received: by mail-vs1-f42.google.com with SMTP id a202so2623163vsd.5 for ; Wed, 24 Oct 2018 01:26:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=/3TxAjY+UmABftiIOrZDg2J636wThikK3HRTnkE1zm8=; b=ngTtDfKUlVozeWpxDMnefNE6IPC43GqruXGhFVf3tSjjgqcjJS5t6WTqr18UgJbeSs C0Juu5deUm93To4niCq5DzEu0XoiiY2ju/KzFgMkS/9oklWA5MsxFyAbypKHi2gkPqfz 0K4P0ZkNE18A5FQtyUQFN46jJazGNyaRzun+ZE92UVNoaHIJfBEgk3UMjkpcLdd9bB8C ejenFTUaUyHBqlgh4Jejk+cUwenVZFqdjAbWLxo07Y2xcZFxvC3N3IaNbTruOOov0U2a 6r1H2Pzt6eaio/pniD1F7B7H7IZrw7xxapsPZio5GcCUf1r2Vx0N/+MgFMAw41ku+THY HARg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=/3TxAjY+UmABftiIOrZDg2J636wThikK3HRTnkE1zm8=; b=Qm+TYWJ04wBxVqglQI/dBKTpVQxNefSgRj3EsVzyIeucdxnxw5waUxNE9prJQQWo8m y1QIJXqYDpEIRencVgYpHFiwxH8HvMXIYO4VQBh/yWqJyYUn/UF8J2vHExGh0+MEow5r C0EL8RLkvyRlhOZ1qOK6U5CU9NbzTZq2Mwmyj7gD5XMQTpLfO0l8L8ghetEK5US8oLqG FzxGKchqZOthGeoXodJEe7XSyAtbBIj6R0jKNSF1si6rakZ+swvwSj3R0ixXO4o3UccW ki0O2j985W+Oo+PpkQJ/s5BwQNQzscG8+o91sS3dn6N/c2S/BRhEarsHh3bTMVz3hDd5 kTOg== X-Gm-Message-State: AGRZ1gKvxQsYr7M7xCJZ9hJ7MP+RhUMFcKUq6dNfYpjXaNydIVbNNHU0 EtV51xy+DTcwV/5S/Lfh+ZGpaxWLNqWGcRX3glYvef3M X-Google-Smtp-Source: AJdET5fbXK/U40EvWmfkhdduUZGCog20POeA+3erTTKUZt7WBJ/pUhhhPoH8Mh958dqgxYDu1OlUbIiyIBoUpBsXetY= X-Received: by 2002:a67:1205:: with SMTP id 5mr689641vss.207.1540369554163; Wed, 24 Oct 2018 01:25:54 -0700 (PDT) MIME-Version: 1.0 References: <1540321728946-0.post@n4.nabble.com> In-Reply-To: From: Andrey Mashenkov Date: Wed, 24 Oct 2018 11:25:42 +0300 Message-ID: Subject: Re: Pre-touch for Ignite off-heap memory To: dev Content-Type: multipart/alternative; boundary="000000000000194d1c0578f53c1a" --000000000000194d1c0578f53c1a Content-Type: text/plain; charset="UTF-8" Docs [1] says, that OOM can also be thrown when native library can't allocate memory chunk if physical memory (ram + swap). >> Shouldn't OOM happen if you're trying to allocate beyond the virtual memory capacity (beyond X GB)? With 64-bit addressing you have some exabytes of virtual memory, is it possible to have such amount of physical memory? I'd think JVM process should be killed by OS much earlier due to memory overcommit. See [2]. [1] https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html [2] http://engineering.pivotal.io/post/virtual_memory_settings_in_linux_-_the_problem_with_overcommit/ On Wed, Oct 24, 2018 at 7:12 AM Denis Magda wrote: > Alex, > > Correct me if I'm wrong, but even if an OS runs out of physical memory (X > GB in total) an Ignite node process still can request the X GB from virtual > memory. Yes, virtual memory can involve swapping and disk to satisfy your > request but this shouldn't be a reason of the OOM. Shouldn't OOM happen if > you're trying to allocate beyond the virtual memory capacity (beyond X GB)? > > Denis > > On Tue, Oct 23, 2018 at 12:08 PM Gerus wrote: > > > Hi *Igniters*, > > Some time ago I've raised a suggestion for product improvement > > https://issues.apache.org/jira/browse/IGNITE-9112 > > . It's all about > > off-heap memory allocation. Current implementation can have some > > improvements for failure critical systems. Ignite can have OOM in > runtime, > > because RAM can be used by OS, if it will not be pre-booked by operation > > system and this proposal is to address that. Common case is offheap and > > thats why memory segment cannot be managed by JVM that has > +AlwaysPreTouch > > option > > Obviously this implementation will make startup longer and thats why it > is > > proposed to use configuration flag to manage this feature > > I think, it will be useful to have this option. Are you supporting this? > > > > > > > > -- > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > -- Best regards, Andrey V. Mashenkov --000000000000194d1c0578f53c1a--