Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 72219200BAA for ; Thu, 27 Oct 2016 21:06:04 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 70B88160AF6; Thu, 27 Oct 2016 19:06:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B7C3A160AE6 for ; Thu, 27 Oct 2016 21:06:03 +0200 (CEST) Received: (qmail 66612 invoked by uid 500); 27 Oct 2016 19:06:03 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 66590 invoked by uid 99); 27 Oct 2016 19:06:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 27 Oct 2016 19:06:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 3B6861A9484 for ; Thu, 27 Oct 2016 19:06:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.398 X-Spam-Level: ** X-Spam-Status: No, score=2.398 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 9hfrgvP8z8fH for ; Thu, 27 Oct 2016 19:06:00 +0000 (UTC) Received: from mail-qk0-f171.google.com (mail-qk0-f171.google.com [209.85.220.171]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id B254D5FC2B for ; Thu, 27 Oct 2016 19:05:59 +0000 (UTC) Received: by mail-qk0-f171.google.com with SMTP id o68so59313867qkf.3 for ; Thu, 27 Oct 2016 12:05:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=b/9jBvQ26nRk0TQm4kwerybpjRnsLUhZMb0J3K79hXE=; b=JBV2Kv2vCEWJr7OqsQbkIi9AfLIVOVKegT+loY94ht0BgVDosbzC4ZLEVfVQ4auSLD xTO2hdbrqpy9lKlqxlW6KcztEvSWj4NMaYE661aVLhByv/dZH7JNu1RyydfTeomdTCTe mS22+Ztucugnfgb6lKyWz2ilQb/1h1dzV8KRLmOpm1Gfnz/DqJ9yah8cz5mPgxAiSn3+ WDmGnHCu1H0jtNTs+QTiOPuvVgyLE7omwFSy4nJpTlQz+XJv1kEnECHWQbLAk2e14VgW B2MrMPi8iiGRO5X4CcMDjkDlUS8bpiWxiMpOpGLajRbv4F5boANFt6mVoL8sJzEkVL+0 ioVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=b/9jBvQ26nRk0TQm4kwerybpjRnsLUhZMb0J3K79hXE=; b=J24Wv1yeZ/rr8qszhA0/+sX72Ohz3E554DYpS9YP5NA7CNUEU59qGfo1wicpS1nK1K PVBfeROXz44PbNbLzylLNA08gKZhNMAdFvPVZgTVFwW/+pD6x+Awq85378bFBHgEPBBp jIudh8t8m72EFkotgbdfI0gpsxaRqdg5Wx8Xhw5nV8v0Ax2uaIWmjAYPMs1T2be7F9Ry nmJUgL0nWj9o/AiW9Gzga+yM0XNHm81cbGsvLWIKEq602wlQCo61c0/0zjZO7dv3Iu+0 e47YpuPB0TdPfVsPsETNcF9tYffoIlMDXsOo5eRmjRUTKMGtYPxyIM+X3dPi1B/JNfAF AJdQ== X-Gm-Message-State: ABUngvcM92mFrjpIgB8CtGvn+co22NqxUkvjraNVrWYiRMeOQ62kTFeHUAFBSbkmJtXoeXUy/Hh0GakTyjy4AA== X-Received: by 10.55.70.77 with SMTP id t74mr8225097qka.78.1477595143075; Thu, 27 Oct 2016 12:05:43 -0700 (PDT) MIME-Version: 1.0 Received: by 10.200.47.115 with HTTP; Thu, 27 Oct 2016 12:05:12 -0700 (PDT) In-Reply-To: References: From: Valentin Kulichenko Date: Thu, 27 Oct 2016 12:05:12 -0700 Message-ID: Subject: Re: Making Ignite shmem library optional To: dev@ignite.apache.org Content-Type: multipart/alternative; boundary=001a114ab570a00d49053fdd6d55 archived-at: Thu, 27 Oct 2016 19:06:04 -0000 --001a114ab570a00d49053fdd6d55 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Absolutely agree. This functionality is unreliable right now and at the very least it should be switched off by default as soon as possible. Going forward, it should be either fixed or removed, because it's just a broken piece of code. -Val On Thu, Oct 27, 2016 at 11:53 AM, Denis Magda wrote: > Igniters, > > Presently, if someone starts up a cluster and has at least two nodes > running on a single Unix machine then those nodes will be communicating > over the shared memory (shmem) by default. > > This approach sounds absolutely reasonable for me but the shmem library i= s > not ideal at the moment. I observed many situations when a cluster got > stuck in the production or during long running tests due to some unclear > issues in shmem internals. Even from Ignite community side we have the > following shmem related issues > https://issues.apache.org/jira/browse/IGNITE-1578 < > https://issues.apache.org/jira/browse/IGNITE-1578> > https://issues.apache.org/jira/browse/IGNITE-1294 < > https://issues.apache.org/jira/browse/IGNITE-1294> > > My proposal would be to make this library optional starting from the > nearest Apache Ignite release. As far as I recall, it=E2=80=99s still man= datory > because it's supposed that it speeds up our Spark and Hadoop integrations= . > But I would avoid using something that can accelerate a solution and fail > at some unpredictable time. > > Any thoughts or controversial arguments? > > =E2=80=94 > Denis > > --001a114ab570a00d49053fdd6d55--