From dev-return-32304-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Wed Mar 21 00:19:00 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id E55CB18064A for ; Wed, 21 Mar 2018 00:18:59 +0100 (CET) Received: (qmail 29215 invoked by uid 500); 20 Mar 2018 23:18:58 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 29197 invoked by uid 99); 20 Mar 2018 23:18:58 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Mar 2018 23:18:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id AD2E71A178A for ; Tue, 20 Mar 2018 23:18:57 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.02 X-Spam-Level: X-Spam-Status: No, score=-0.02 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=trimble-com.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id EYIT3mfnm9HL for ; Tue, 20 Mar 2018 23:18:56 +0000 (UTC) Received: from mail-wm0-f45.google.com (mail-wm0-f45.google.com [74.125.82.45]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 009535FB70 for ; Tue, 20 Mar 2018 23:18:55 +0000 (UTC) Received: by mail-wm0-f45.google.com with SMTP id f19so6564512wmc.0 for ; Tue, 20 Mar 2018 16:18:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=trimble-com.20150623.gappssmtp.com; s=20150623; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to; bh=j52GV0Wfj5n2r/34PU04K6hbztZ/R0tPo68RMCDG+8M=; b=ez5TF+44r2dmhP9cj1U3SuJ70gC1rMbiqjShi6MUVs2fStuiNsCR1LN7uOs1fsJ9B9 gO3P1CWkM2/jw53PQJiEyVQpwOKWJww/z8Vx4JbNFhPX63Ekvvw73sr86ppgiW41sABi JWgi0FF4rEQ3VLahE8GB9E6s1AmNnbElwqkzwzbpD4zpd6VYTTMzV7ahC/VktH6erhff wTrpUIHCkmQquz+78pYH30wPX0fPVLhlOCQOQsSL20UPSTEzLQNf76aYyqxtGy7XxU1R +5zo85HKM5JaY7Tp+2WO8kSH40b2wkSq+cEYj9yJTwEc88MvUtJ2Ckff/lLvIhUuBvng jMaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to; bh=j52GV0Wfj5n2r/34PU04K6hbztZ/R0tPo68RMCDG+8M=; b=FCxdd4ncf38DANBameHimxrbtuwQff0PiEsyeZjyNfcMM32QP0HAjucJQxxsWrm9Wu sVZOdx183WqKEv3DBp0APQLnlOb72lkq/kvHPo8x7YHS/n9PCet5MHKH/Cr9KqS6ZKsy wnnWEreZQITf6wHx0H780I1b324weEKf0NUITqxEvuVuEiQ1OYwJM8km6RsTWvj+xWDP fNBesTBINqiX7LkV+cMhmHTuZyClQZt5XFEZE60SUc4/wJhaRgr+8JPShlYWonShmQeA viRHKwwNC3FoC3nOI+F6qfsF1R2vgDDIJrjwLT9/quJxJX1rNS1KP1kL85tT82oWdrch xvfQ== X-Gm-Message-State: AElRT7H68Sz8cBcif7Uwb8gSVODjE+5br/RgGc9v0c8/1KL0+83JcIX8 rPdS9mEnSBrlZvH81Lz764PtBZ2mwdj7TXGUCtzLfsh5 X-Google-Smtp-Source: AG47ELvt68vOKLBHQTnThzC2pz6Ozwn1nygV/kFdAnBfSJLV9ozizXmTOKtOQx6G4UiSankI3h3q5GN/ciAIMEmXpv8= X-Received: by 10.80.207.75 with SMTP id d11mr18842768edk.119.1521587934358; Tue, 20 Mar 2018 16:18:54 -0700 (PDT) From: Raymond Wilson References: <44857f780437831523c7ab8515802df3@mail.gmail.com> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQF4c89nzo4tQ2KXK+8azsg7c3EtQgHG23a+AVycF0QBD44gCKRuwyTw Date: Wed, 21 Mar 2018 12:18:47 +1300 Message-ID: <48564967d71e6e7898b6c04f97f4b4ea@mail.gmail.com> Subject: RE: Ignite Direct I/O plugin description added to wiki To: dev@ignite.apache.org, Prachi Garg Content-Type: text/plain; charset="UTF-8" Denis, Thanks for the clarification on read-intensive workloads not being suitable for Direct IO. As a counterpoint to that, I would have thought the working set represented by the in-memory data held by the Ignite node would be significantly larger than the OS page cache which should mean the read hit rate on the OS page cache would be very low under those workloads? Of course, this assumes your WAL page size is configured to be similar to the OS page size, but still.... -----Original Message----- From: Denis Magda [mailto:dmagda@apache.org] Sent: Wednesday, March 21, 2018 11:59 AM To: dev@ignite.apache.org; Prachi Garg Subject: Re: Ignite Direct I/O plugin description added to wiki *Dmitriy*, thanks. Astonishing job! We'll add a section to the durable memory tuning page and refer to the wiki for more details: https://issues.apache.org/jira/browse/IGNITE-7466 Please clarify the following: > Direct I/O mode can't be enabled for Write Ahead Log files. However, > when working with plugin, WAL manager applies advising Linux systems > do not store the data of the file in page cache as they are not required. For me, it means that WAL always goes through the operating system I/O calls. Nothing changes for the WAL. However, I'm not sure what you meant to explain by saying "when working with the plugin (Direct I/O) WAL manager applies...". Could you rephrase it to bring more clarity? *Raymond,* If Direct I/O is enabled by default it will bring down the performance of read-intensive application because, as Dmitry says, the reads bypass page cache. So, I would recommend using it for write-intensive workloads and, probably, for mixed-workloads depending on the reads and writes rate. -- Denis On Tue, Mar 20, 2018 at 2:29 PM, Raymond Wilson wrote: > Looks good! > > Is there any reason why this should not be a default setting if it > gracefully downgrades to non-Direct IO if not supported by the OS? > > Thanks, > Raymond. > > -----Original Message----- > From: Dmitriy Setrakyan [mailto:dsetrakyan@apache.org] > Sent: Wednesday, March 21, 2018 10:23 AM > To: dev > Subject: Re: Ignite Direct I/O plugin description added to wiki > > Thanks Dmitry, awesome work! > > On Wed, Mar 21, 2018 at 12:21 AM, Dmitry Pavlov > > wrote: > > > Hi Igniters, > > > > I've added description of new plugin for Direct I/O for native > > persistence ( > > https://issues.apache.org/jira/browse/IGNITE-6341) to wiki > > https://cwiki.apache.org/confluence/display/IGNITE/ > > Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore- > > underthehood-DirectI/O > > > > > > SIncerely, > > Dmitriy Pavlov > > >