Return-Path: X-Original-To: apmail-ignite-user-archive@minotaur.apache.org Delivered-To: apmail-ignite-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9C71218CD5 for ; Fri, 15 Jan 2016 07:28:14 +0000 (UTC) Received: (qmail 24568 invoked by uid 500); 15 Jan 2016 07:28:14 -0000 Delivered-To: apmail-ignite-user-archive@ignite.apache.org Received: (qmail 24520 invoked by uid 500); 15 Jan 2016 07:28:14 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 24510 invoked by uid 99); 15 Jan 2016 07:28:14 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Jan 2016 07:28:14 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id EF09FC09EC for ; Fri, 15 Jan 2016 07:28:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.901 X-Spam-Level: ** X-Spam-Status: No, score=2.901 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id re0GWltwefSW for ; Fri, 15 Jan 2016 07:28:02 +0000 (UTC) Received: from mail-qg0-f53.google.com (mail-qg0-f53.google.com [209.85.192.53]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 03C1C43F3A for ; Fri, 15 Jan 2016 07:28:02 +0000 (UTC) Received: by mail-qg0-f53.google.com with SMTP id 6so423379320qgy.1 for ; Thu, 14 Jan 2016 23:28:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=3aqJNh43yfzqeeYViZBHsWd1+ug4DoGRu4STnj625j4=; b=muiPoctPP0iX+hgKk8WkxBb9yrEcxw8OLzdSeEni621TljEuV1ZnfiweNLxlWd7R5q Xu9wjmW4BHLcNjaQYD+ppFVM1Ld/31NrFQbFQ5g2eBgoVOHjZ7IBjnFoa7h2F4cCEvpC TJIUZcQUfz/bqg4xzcqJY/+/6BOy8v6dbi5vOUZHKO15hWgHkl0ieM7JRZUoEZlpPLaV rHSd5EnueXl8IpPOjJq3zt0jrrwl6cG/1l5dMfhii9qRBAC8XmRzPy+781ToOGn3FIR4 8Q6jugv6wDXqijdPXfarbn4xThwmI9/xL6svBrovQryukNGR33/ucHVn6Yiid95NHbM9 n0hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=3aqJNh43yfzqeeYViZBHsWd1+ug4DoGRu4STnj625j4=; b=hMAEdXRooLBKOkBAgo9YoYox005Ax+nvfg1Zc2MaK1aGG5f12a5IcHiXI09iuAMpJT 4WeZnCAQ22/lyIjo3+0X4fHgMB1DotEJsZTabXC1EEcWLFDYSFe7xU7uqQVZaFFwnpCb LieoC5POGafBkfiXDQWLE0jteIQGedsghNyEoCSbrmu1Jw0YO7ixhjS5drhLx2isoY6g +s2ag4G2guhWg6dXxvjLmMrE72589iuDZFqMLYSoMqK6w6MLymZbLfrmdA9t22CIE3gz t1cxjjoIHj6jKhUrhglxbuQrleSn1OBXB1dVZv7aYzsMPyySQIN71oBf98iRYmdidW0w 1Kqw== X-Gm-Message-State: ALoCoQmCadqf2gA5Uq9EVtlPAsaEeJ1U57aDdd3KHQ1bNvuit5cOFvJNSkOcKe7z6ggSUPcDMeHPyLBiLVCo35q1tgzlbXduyw== MIME-Version: 1.0 X-Received: by 10.140.128.8 with SMTP id 8mr11665104qha.54.1452842875359; Thu, 14 Jan 2016 23:27:55 -0800 (PST) Received: by 10.140.25.195 with HTTP; Thu, 14 Jan 2016 23:27:55 -0800 (PST) In-Reply-To: References: <62AB8A812A5FAB4EA63EBC8E73952242DBD8F3@PARS1PZEXMBX007.mercury.intra> <5697A64A.9080107@gridgain.com> Date: Thu, 14 Jan 2016 23:27:55 -0800 Message-ID: Subject: Re: Stupide question about Queue ? From: Valentin Kulichenko To: user@ignite.apache.org Content-Type: multipart/alternative; boundary=001a1134f18e801be605295a570a --001a1134f18e801be605295a570a Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Murthy, Swap space allows to write evicted data to disc to avoid out of memory. This is not persistence, because this data will be lost if the node leaves topology. For caches persistence can be implemented with the help of CacheStore interface [1]. For queues and sets it's not possible now, I will start a separate thread on dev@ about this. [1] https://apacheignite.readme.io/docs/persistent-store -Val On Thu, Jan 14, 2016 at 11:16 PM, Valentin Kulichenko < valentin.kulichenko@gmail.com> wrote: > Created ticket for queue swapping feature: > https://issues.apache.org/jira/browse/IGNITE-2389 > > Feel free to provide feedback. > > -Val > > On Thu, Jan 14, 2016 at 3:24 PM, Murthy Kakarlamudi > wrote: > >> What are the possibilities for swap spaces? Can we persist the >> cache/queue content to file system based on the eviction policy? >> >> Thanks, >> Murthy. >> >> On Thu, Jan 14, 2016 at 5:23 PM, Valentin Kulichenko < >> valentin.kulichenko@gmail.com> wrote: >> >>> Agree with Denis. Sounds like this can be easily achieved by enabling >>> swap layer for queue cache. So we just need to put this on configuratio= n. >>> >>> -Val >>> >>> On Thu, Jan 14, 2016 at 5:44 AM, Denis Magda >>> wrote: >>> >>>> Hi, >>>> >>>> This seems to be an interesting functionality. As I understand you're >>>> referring to [1]. >>>> >>>> Personally I think that it makes sense to support the same in Ignite. >>>> However I would just extend CollectionConfiguration adding ability to >>>> set an eviction policy and enable swap on demand. When eviction policy >>>> faces with a limit entries will be evicted to swap space. >>>> Don't think that we need to support other kind of storages here. >>>> >>>> Igniters, do you have other thoughts on this? >>>> >>>> [1] >>>> http://docs.hazelcast.org/docs/3.5/manual/html/queue-persistence.html >>>> >>>> -- >>>> Denis >>>> >>>> >>>> On 1/14/2016 1:41 PM, Yann BLAZART wrote: >>>> >>>> Hello, >>>> >>>> >>>> >>>> In hazelcast we can persist Queues in Database, this help me for >>>> example when consumer are down to no have OOM exception. >>>> >>>> >>>> >>>> But I don=E2=80=99t see this possibility in Ignite docs ? >>>> >>>> >>>> >>>> This message and any attachments (the "message") is >>>> intended solely for the intended addressees and is confidential. >>>> If you receive this message in error,or are not the intended >>>> recipient(s), >>>> please delete it and any copies from your systems and immediately noti= fy >>>> the sender. Any unauthorized view, use that does not comply with its >>>> purpose, >>>> dissemination or disclosure, either whole or partial, is prohibited. >>>> Since the internet >>>> cannot guarantee the integrity of this message which may not be >>>> reliable, BNP PARIBAS >>>> (and its subsidiaries) shall not be liable for the message if modified= , >>>> changed or falsified. >>>> Do not print this message unless it is necessary,consider the >>>> environment. >>>> >>>> >>>> ----------------------------------------------------------------------= ------------------------------------------------------------ >>>> >>>> Ce message et toutes les pieces jointes (ci-apres le "message") >>>> sont etablis a l'intention exclusive de ses destinataires et sont >>>> confidentiels. >>>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine, >>>> merci de le detruire ainsi que toute copie de votre systeme et d'en >>>> avertir >>>> immediatement l'expediteur. Toute lecture non autorisee, toute >>>> utilisation de >>>> ce message qui n'est pas conforme a sa destination, toute diffusion ou >>>> toute >>>> publication, totale ou partielle, est interdite. L'Internet ne >>>> permettant pas d'assurer >>>> l'integrite de ce message electronique susceptible d'alteration, BNP >>>> Paribas >>>> (et ses filiales) decline(nt) toute responsabilite au titre de ce >>>> message dans l'hypothese >>>> ou il aurait ete modifie, deforme ou falsifie. >>>> N'imprimez ce message que si necessaire, pensez a l'environnement. >>>> >>>> >>>> >>> >> > --001a1134f18e801be605295a570a Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Murthy,

Swap space allows to write e= victed data to disc to avoid out of memory. This is not persistence, becaus= e this data will be lost if the node leaves topology. For caches persistenc= e can be implemented with the help of CacheStore interface [1]. For queues = and sets it's not possible now, I will start a separate thread on dev@ = about this.


-Val

On Thu, Jan 14, 2016 at 11:16 PM, Valen= tin Kulichenko <valentin.kulichenko@gmail.com> w= rote:
Created ticket for queue sw= apping feature: https://issues.apache.org/jira/browse/IGNITE-2389<= /div>

Feel free to provide feedback.

-Val

On Thu, Jan 14, 2016 at 3:24 PM, Mur= thy Kakarlamudi <ksatya@gmail.com> wrote:
What are the possibilities for swap spaces? Can we persis= t the cache/queue content to file system based on the eviction policy?
=
Thanks,
Murthy.

On Thu, Jan 14, 2016 at 5:23 P= M, Valentin Kulichenko <valentin.kulichenko@gmail.com><= /span> wrote:
Agree with Denis. Sounds= like this can be easily achieved by enabling swap layer for queue cache. S= o we just need to put this on configuration.

-Val
<= /div>

On Thu, Jan = 14, 2016 at 5:44 AM, Denis Magda <dmagda@gridgain.com> wro= te:
=20 =20 =20
Hi,

This seems to be an interesting functionality. As I understand you're referring to [1].

Personally I think that it makes sense to support the same in Ignite.
However I would just extend CollectionConfiguration adding ability to set an eviction policy and enable swap on demand. When eviction policy faces with a limit entries will be evicted to swap space.
Don't think that we need to support other kind of storages here.
Igniters, do you have other thoughts on this?

[1] http://docs.hazelcast.org/docs/3.5/manual/html= /queue-persistence.html

--
Denis


On 1/14/2016 1:41 PM, Yann BLAZART wrote:
=20 =20 =20

Hello,

=C2=A0

In hazelcast we can persist Queues in Database, this=C2=A0 help me for example when consumer are down to no have OOM exception.

=C2=A0

But I don=E2=80=99t see= this possibility in Ignite docs ?

=C2=A0

This message and any attachments (the "message") is
intended solely for the intended addressees and is confidential.
If you receive this message in error,or are not the intended recipient(s),
please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its purpose,
dissemination or disclosure, either whole or partial, is prohibited. Since the internet
cannot guarantee the integrity of this message which may not be reliable, BNP PARIBAS
(and its subsidiaries) shall not be liable for the message if modified, changed or falsified.
Do not print this message unless it is necessary,consider the environment.

------------------------------------------------------------------= ----------------------------------------------------------------

Ce message et toutes les pieces jointes (ci-apres le "message= ")
sont etablis a l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de
ce message qui n'est pas conforme a sa destination, toute diffusion ou toute
publication, totale ou partielle, est interdite. L'Internet ne permettant pas d'assurer
l'integrite de ce message electronique susceptible d'altera= tion, BNP Paribas
(et ses filiales) decline(nt) toute responsabilite au titre de ce message dans l'hypothese
ou il aurait ete modifie, deforme ou falsifie.
N'imprimez ce message que si necessaire, pensez a l'environnement.






--001a1134f18e801be605295a570a--