From user-return-19109-archive-asf-public=cust-asf.ponee.io@ignite.apache.org Thu Apr 26 20:34:46 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id BA41C180648 for ; Thu, 26 Apr 2018 20:34:45 +0200 (CEST) Received: (qmail 89226 invoked by uid 500); 26 Apr 2018 18:34:44 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 89212 invoked by uid 99); 26 Apr 2018 18:34:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 26 Apr 2018 18:34:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 0F12DC0148 for ; Thu, 26 Apr 2018 18:34:44 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.001 X-Spam-Level: X-Spam-Status: No, score=-0.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=trimble-com.20150623.gappssmtp.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id W4uwNJzsSJoO for ; Thu, 26 Apr 2018 18:34:42 +0000 (UTC) Received: from mail-pf0-f178.google.com (mail-pf0-f178.google.com [209.85.192.178]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 22A355F666 for ; Thu, 26 Apr 2018 18:34:42 +0000 (UTC) Received: by mail-pf0-f178.google.com with SMTP id q22so4106951pff.11 for ; Thu, 26 Apr 2018 11:34:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=trimble-com.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:date:subject:message-id :references:in-reply-to:to; bh=9wy06eDi7azJwz+lQTfiNAYBcOSGBHzsonrbYNXM/Mc=; b=1IMwClp6fUd2qGre9hpMJdCMd9VjiwyO35Hza5tCOWpIBTQb3dbDfdhDabAx2Q424d iQRSd7e2z+PDNOM8f64S1sv6QZ+qlqFy0VP/jdfzfM9bYv0/AajIijBAL8zaV8Kp3MDh JidD27BROjOuR2ae1miG5+WF0LPmynKmLe9aSMXhJ0uxPO481Co+uUawkHuDwbZoLbKq Obtk5/LzrpL0X74ty0w5ow4fW+RyBHAyDoKISqu32y/B3OPuXuDTHPSQLisUXLvLESWS PKfiGhKK//UvD/+QMoBf40AJoYLxe86AAB2dPraNdiZht4JmNtPimoZsLjqNLudtPAOi yYmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version:date :subject:message-id:references:in-reply-to:to; bh=9wy06eDi7azJwz+lQTfiNAYBcOSGBHzsonrbYNXM/Mc=; b=ZVAeXza2FLi0NR9AooVuse+IbKe2WH/uZ6ke//MfPbxZ15mrlmUZqWU+qnGSksDHL9 P1qgUclpJ3RVlvPOD52qdfQEiH1JAuYWcTZAvKxblQcYdoE1fafpvvSxdkriWkXkdaSc qNc54w3iQAyYZDNaT/MmxJYB+8rJIIeMBitXh/aui/ZK5ZWe8RJQLIfA5od8EYTkFuyM XhxWSfA9XDMRiMAD50AZr3Wq/hlWfK+/gPBsHVTuK2ci4gSDeJaAzibJmo/U1SWBwrv6 n+GRlVAnWJyZkTLyXbMcY6dg0/Pj50SoVzjkWJ+rbwRp4Ml9Ni2xe5kM0+cUL0iJKuGf st8w== X-Gm-Message-State: ALQs6tCfrsE71FT7P0xHz/jOvi0MfUFqqP2MKPIRQ0pW+QWTbQaLktXC dK1K+i0ixBqt9SYMHGpSJnXkaDc6cu0= X-Google-Smtp-Source: AB8JxZp2EeUAkH9atIa7hA9VtQaOdl6arjzvJZPikyycFiThLhDxS5p7ub6Tog4wtP02AUAxiq17zw== X-Received: by 10.101.81.131 with SMTP id h3mr8228803pgq.58.1524767679633; Thu, 26 Apr 2018 11:34:39 -0700 (PDT) Received: from [192.168.178.35] (163.29.255.123.dynamic.snap.net.nz. [123.255.29.163]) by smtp.gmail.com with ESMTPSA id q25sm33622359pff.134.2018.04.26.11.34.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Apr 2018 11:34:38 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable From: Raymond Wilson Mime-Version: 1.0 (1.0) Date: Fri, 27 Apr 2018 06:22:31 +1200 Subject: Re: Using a cache as an affinity co-located processing buffer in Ignite.Net Message-Id: References: <69bc1ca111f81662a7a2ec175537367c@mail.gmail.com> <4BA7D21C-0389-4A49-9027-E47C9AFC476C@trimble.com> <11756955367caaf370818143b23ba2dc@mail.gmail.com> <0dfa951f43b9273912a3df9ddce3c4d0@mail.gmail.com> <1524764955581-0.post@n6.nabble.com> In-Reply-To: <1524764955581-0.post@n6.nabble.com> To: user@ignite.apache.org X-Mailer: iPhone Mail (14G60) Val, What I really want is a .Net implementation of the Ignite distributed queue,= assuming co-located server processing can pull members off the queue that a= re co-located with that node (yeah, that's probably mangling queue semantics= a bit, but not that badly :) ) I have previously made a POC code implementation where a service is deployed= to every node and runs a continuous query against the local members of the c= ache (this one does not abuse remote filters). Ie: one CQ per node. This app= roach requires a single context to issue the service deployment (which goes b= ack to the best practices question) I have a second implementation where a single context issues the CQ and reli= es on remote filters to deal with the cache entries. To be fair, the remote f= ilter just shunts the item into another class which asynchronously processes= them (so I don't think there is a significant locking issue), though this i= ntroduces the need for a static singleton to handle the dual remote filter c= ontexts for the scan query and continuous query aspects of the CQ. The exactly once guarantee differences are food for thought. The processing i= s idempotent, but still not desirable to process things more than once if no= t needed.=20 Sent from my iPhone > On 27/04/2018, at 5:49 AM, vkulichenko wro= te: >=20 > Raymond, >=20 > If you want to have a single continuous query on one of the server nodes, > then I think singleton service is the best option to achieve that. It will= > not only guarantee that there is one query at a time, but will also make > sure to redeploy it in case of failure. >=20 > Also I would be accurate with doing processing in the remote filter. Filte= r > is not designed for this, it's designed to do the filtering of updates > before sending them to listening node. Major pitfall here is that it's > invoked within entry lock, so any heavy processing would block the entry. > Moreover, synchronous cache or other operations can cause deadlocks and/or= > thread starvation. Finally, keep in mind that filter can be invoked multip= le > times (at least for primary and backups, sometimes even more in case of > failures). There is no exactly-once guarantee for remote filter, it's only= > applied to local listener. >=20 > -Val >=20 >=20 >=20 > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/