Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 68D4D200CA6 for ; Wed, 14 Jun 2017 00:00:19 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 67461160BDC; Tue, 13 Jun 2017 22:00:19 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 0FB12160BC5 for ; Wed, 14 Jun 2017 00:00:17 +0200 (CEST) Received: (qmail 29912 invoked by uid 500); 13 Jun 2017 22:00:17 -0000 Mailing-List: contact user-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@ignite.apache.org Delivered-To: mailing list user@ignite.apache.org Received: (qmail 29901 invoked by uid 99); 13 Jun 2017 22:00:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Jun 2017 22:00:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id BCF821AFC1F for ; Tue, 13 Jun 2017 22:00:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.794 X-Spam-Level: *** X-Spam-Status: No, score=3.794 tagged_above=-999 required=6.31 tests=[AC_DIV_BONANZA=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, URI_HEX=1.313] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=trimble-com.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id Ylt3oX4jzhdO for ; Tue, 13 Jun 2017 22:00:14 +0000 (UTC) Received: from mail-qt0-f171.google.com (mail-qt0-f171.google.com [209.85.216.171]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id E8CE25F3F0 for ; Tue, 13 Jun 2017 22:00:13 +0000 (UTC) Received: by mail-qt0-f171.google.com with SMTP id c10so191189144qtd.1 for ; Tue, 13 Jun 2017 15:00:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=trimble-com.20150623.gappssmtp.com; s=20150623; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to; bh=Tp5Kkday3vLYjHLs3/SYQBUJlSJypRwPchW5O5ty02A=; b=m/M52OQN2vwl9WXd8kqYhM6qEG3HNq9V79NSjjQw9LI7p46E35mA73yVpav/ECbptt +2Ou/INDPoqtHz61cBndEzvrSRC5vrCcfsGCjQ8YlSbyIaEtDB1cvdo312WPMmaGNrt6 DU/P6wqCwnjLPh3ecM82g7RRUsqt30/Bo9uRX6/w+DlfXPDypqJhgEaQCvmG96TAfq9Y n+/rPxRz3H/8ghPvRe998vbMCERctT1wbBBfStlgZF254cc5sIb810WCtLh3U4jdWi3e WuJnsejV24taGymX/NRuMjFYFHG9vHMRmYpBmlUTEClQiNRXBx/aPDkUNmo7IJDzXmDB azSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to; bh=Tp5Kkday3vLYjHLs3/SYQBUJlSJypRwPchW5O5ty02A=; b=i8GYRNUZ7H7o8FZwHF/7YdRmUV1D7Uz2yvpTTkbeC5epVu0TStdmR0lb09EVo//93k omywI+LloDOIAUxskQ6WXJSyzLDm4GnHetfE6slVRjxMw49It87K0d9ZX/EqwKDMrxw7 OfRRBLUMixxo4W0L/uza9/Un3R10T2uAKNXL5lToB6Af4XU79w9gs4wCCnA4tgNR4l4m /vN988eMBmuHAruAdyFE0whUqt2GhUXZ4uZgW2XVsNHnVNJjAMpLw9e3vb9wi5DPeEGp W8qeZzpJwxwE3u4Rx++9Q1FBJFqnoKlRTHeHjA/FypNII2cs74jl1AyJPGVo+hOHNrbh CWoA== X-Gm-Message-State: AKS2vOzWOAvr4HGIHtfDsn2KLz5ksltl9vXRIIurIulGsYWVpkdPhrKU E9zX+A99WfBwmr+L3vuS7Y+M5xrXB5ZtzDc= X-Received: by 10.237.34.148 with SMTP id p20mr2895447qtc.90.1497391211631; Tue, 13 Jun 2017 15:00:11 -0700 (PDT) From: Raymond Wilson References: <72329982ff4c8841916ccb87a93a21b7@mail.gmail.com> <271CA4A4-A114-4FF4-A2E5-E6495CFE29FF@apache.org> In-Reply-To: <271CA4A4-A114-4FF4-A2E5-E6495CFE29FF@apache.org> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQGBRleBkR5Sp6Ygmx7vrSAty7DoJwJDBl2VAd0/noEB0kwTQgEXbVXwAn41NoABRiBcigIoiPbJAgswKnYC8Z8l8QIbFvqFAbsx42SiGERUAA== Date: Wed, 14 Jun 2017 10:00:10 +1200 Message-ID: Subject: RE: Write behind using Grid Gain To: user@ignite.apache.org Content-Type: multipart/alternative; boundary="001a113d1a4242c1670551de8f56" archived-at: Tue, 13 Jun 2017 22:00:19 -0000 --001a113d1a4242c1670551de8f56 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Denis, Ah! Looks very interesting. Thanks for the pointer J Raymond. *From:* Denis Magda [mailto:dmagda@apache.org] *Sent:* Wednesday, June 14, 2017 9:41 AM *To:* user@ignite.apache.org *Subject:* Re: Write behind using Grid Gain Raymond, Then Ignite Persistent Store is exactly for your use case. Please refer to this discussion on the dev list: http://apache-ignite-developers.2346864.n4.nabble.com/GridGain-Donates-Pers= istent-Distributed-Store-To-ASF-Apache-Ignite- td16788.html#a16838 Also it was covered a bit in that webinar: https://www.youtube.com/watch?v=3DbDrGueQ16UQ The store should be released by the community in the nearest. =E2=80=94 Denis On Jun 13, 2017, at 2:02 PM, Raymond Wilson wrote: Hi Pavel, It=E2=80=99s a little complicated. The system is essentially a DB in its ow= n right; actually it=E2=80=99s an IMDG a bit like Ignite, but developed 8 years ago = to fulfill a need we had. Today, I am looking to modernize that system and rather than continuing to build and maintain all the core =E2=80=98infrastructure=E2=80=99 features o= f an IMDG such as clustering, messaging, enterprise caching etc, I am looking to see how well Ignite fits by running a Proof of Concept project. It turns out it fits quite well, largely because the architectural structure of both systems (ie: IMDG) is well aligned in terms of the problems being solved. The primary gap between the legacy system and IMDG is that IMDG does not support persistence. The legacy system has a distributed cache that stores objects that are aggregate collections (10=E2=80=99s of thousand=E2=80=99s)= of relatively simple spatial data records that are operated on by the clustered compute engine. Sometimes billions of records need to be processed to satisfy a single query. Your standard run of the mill SQL DB finds these sorts of queries hard. I suppose you could use another DB (MS-SQL, AWS:RDS etc) to store those aggregate blobs, but it seems like a bit of a =E2=80=98miss-use case=E2=80= =99 when what I=E2=80=99m really after is a persistence/storage layer J Thanks, Raymond. *From:* Pavel Tupitsyn [mailto:ptupitsyn@apache.org] *Sent:* Wednesday, June 14, 2017 1:59 AM *To:* user@ignite.apache.org *Subject:* Re: Write behind using Grid Gain Hi Raymond, I think your use case fits well into traditional Ignite model of write-through cache store with backing database. Why do you want to avoid a DB? Do you plan to store data on disk directly as a set of files? Pavel On Mon, Jun 12, 2017 at 2:14 AM, Raymond Wilson wrote: Hi Pavel, Thanks for the blog =E2=80=93 it explains it quite well. I have a slightly different use case where groups of records within a much larger data set are clustered together for efficiency (ie: each of the cached items in the Ignite grid cache has significant internal structure). You can think of them as a large number of smallish files (a few Kb to a few Mb), but file systems don=E2=80=99t like lots of small files. I have a legacy implementation that houses these small files within a single larger file, but wanted to know if there was a clean way of supporting the same structure using the Ignite read/write through support, perhaps with another system providing relatively transparent persistency semantics but which does not use a DB to store the data. Thanks, Raymond. *From:* Pavel Tupitsyn [mailto:ptupitsyn@apache.org] *Sent:* Saturday, May 27, 2017 5:03 AM *To:* user@ignite.apache.org *Subject:* Re: Write behind using Grid Gain I've decided to write a blog post, since this topic seems to be in demand: https://ptupitsyn.github.io/Ado-Net-Cache-Store/ Code: https://github.com/ptupitsyn/ignite-net-examples/tree/master/AdoNetCacheSto= re Let me know if this helps! On Fri, May 26, 2017 at 3:50 PM, Chetan D wrote: Thank you Pavel. waiting for your response. On Fri, May 26, 2017 at 6:03 PM, Pavel Tupitsyn wrote: To give everyone context, this is not about GridGain, but about Apache Ignite. The blog post in question is https://ptupitsyn.github.io/Entity-Framework-Cache-Store/ Chetan, I'll prepare an example with Ignite 2.0 / ado.net and post it some time later. Pavel On Fri, May 26, 2017 at 2:32 PM, Chetan D wrote: ++ User List any help much appreciated. Thanks And Regards Chetan D ---------- Forwarded message ---------- From: *Pavel Tupitsyn* Date: Fri, May 26, 2017 at 4:38 PM Subject: Re: Write behind using Grid Gain To: Chetan D Hi Chetan, can you please write this to our user list, user@ignite.apache.org? So that entire community can participate. Thanks, Pavel On Fri, May 26, 2017 at 1:35 PM, Chetan D wrote: Hi Pavel Tupitsyn, I have posted a comment in your blog as well (entity framework as ignite .net store) regarding write behind using ignite. I have been working on a project where i need to implement distributed caching and i have been asked to look into grid gain. This is is the first time i am working on caching and this is entirely new topic for me. The example which you have shared i was able to understand a little and the sad part is even entity framework also i have never worked on. It would be helpful if you can share me a simple example using ado.net implementing read through, write through and write behind even a simple table helps me understand the concept. --001a113d1a4242c1670551de8f56 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

De= nis,

=C2=A0

Ah! Looks very int= eresting. Thanks for the pointer J

=C2=A0

Raymond.

=C2=A0

<= p class=3D"MsoNormal">From: Denis Mag= da [mailto:dmagda@apache.org]
= Sent: Wednesday, June 14, 2017 9:41 AM
To: user@ignite.apache.org
Subject: Re= : Write behind using Grid Gain

=C2=A0

Raymond,

= =C2=A0

Then Ignite Persistent Store is= exactly for your use case. Please refer to this discussion on the dev list= :

http://apache-ignite-devel= opers.2346864.n4.nabble.com/GridGain-Donates-Persistent-Distributed-Store-T= o-ASF-Apache-Ignite-td16788.html#a16838

= Also it was covered a bit in that webinar:

https://www.youtube.com/watch?v=3DbDrGueQ16UQ

<= p class=3D"MsoNormal">The store should be released by the community in the = nearest.

=C2=A0

=E2=80=94

Denis

=C2=A0

On Jun 13, 2017, a= t 2:02 PM, Raymond Wilson <raymond_wilson@trimble.com> wrote:

= =C2=A0

Hi Pa= vel,

=C2= =A0

It=E2= =80=99s a little complicated. The system is essentially a DB in its own rig= ht; actually it=E2=80=99s an IMDG a bit like Ignite, but developed 8 years = ago to fulfill a need we had.=C2=A0

=C2= =A0

Today,= I am looking to modernize that system and rather than continuing to build = and maintain all the core =E2=80=98infrastructure=E2=80=99 features of an I= MDG such as clustering, messaging, enterprise caching etc, I am looking to = see how well Ignite fits by running a Proof of Concept project. It turns ou= t it fits quite well, largely because the architectural structure of both s= ystems (ie: IMDG) is well aligned in terms of the problems being solved.

=C2=A0

The primary ga= p between the legacy system and IMDG is that IMDG does not support persiste= nce. The legacy system has a distributed cache that stores objects that are= aggregate collections (10=E2=80=99s of thousand=E2=80=99s) of relatively s= imple spatial data records that are operated on by the clustered compute en= gine. Sometimes billions of records need to be processed to satisfy a singl= e query. Your standard run of the mill SQL DB finds these sorts of queries = hard.

=C2= =A0

I supp= ose you could use another DB (MS-SQL, AWS:RDS etc) to store those aggregate= blobs, but it seems like a bit of a =E2=80=98miss-use case=E2=80=99 when w= hat I=E2=80=99m really after is a persistence/storage layer=C2=A0J

=C2=A0

<= span style=3D"font-size:11.0pt;font-family:"Calibri","sans-s= erif";color:#1f497d">Thanks,

Raymond.

=C2=A0

=C2=A0

From:= =C2=A0Pavel Tupitsyn [mailto:ptupitsyn@apache.org]=C2=A0
Sent:=C2=A0Wednesday, June 14, 2017 1= :59 AM
To:=C2=A0user@ign= ite.apache.org
Subject:=C2=A0Re: Write behind using Grid Gain

=C2=A0

Hi Raymond,

=

=C2=A0

I think your use case fits well into traditional Ignite model of wr= ite-through cache store with backing database.

Why do you want to avoid a DB? Do you plan to store dat= a on disk directly as a set of files?

=C2=A0

Pavel

=C2=A0

On Mon, Jun 12, 2017 at 2:14 AM, Raymond Wilson <ra= ymond_wilson@trimble.com> wrote:

Hi Pavel,

=C2=A0

Thanks for the blog =E2=80=93 i= t explains it quite well.

=C2=A0

I have a slightly different use case where groups of record= s within a much larger data set are clustered together for efficiency (ie: = each of the cached items in the Ignite grid cache has significant internal = structure). You can think of them as a large number of smallish files (a fe= w Kb to a few Mb), but file systems don=E2=80=99t like lots of small files.= =C2=A0

=C2=A0

I have a legacy implementation th= at houses these small files within a single larger file, but wanted to know= if there was a clean way of supporting the same structure using the Ignite= read/write through support, perhaps with another system providing relative= ly transparent persistency semantics but which does not use a DB to store t= he data.

= =C2=A0

Tha= nks,

Raymond.

=C2=A0

From:=C2=A0Pavel Tupitsyn [mailto:ptupitsyn@apache.org= ]=C2=A0
Sent:=C2=A0Saturday, May 27, 2017 5:03 AM<= br>To:=C2=A0user@ignite.apache.org


Subject:= =C2=A0Re: Write behind using Grid Gain

=C2=A0

I'v= e decided to write a blog post, since this topic seems to be in demand:

=

=C2=A0

C= ode:

=

=C2=A0

Let= me know if this helps!

= =C2=A0

On Fri, May 26, 2017 at 3:50 PM,= Chetan D <cchetands@gmail.com> wrote:

Thank you Pavel.

=C2=A0

waiting for= your response.

= =C2=A0

On Fri, May 26, 2017 at 6:03 PM,= Pavel Tupitsyn <ptupitsyn@apache.org> wrote:<= /p>

To give everyone context, this is= not about GridGain, but about Apache Ignite.

=C2=A0

Chetan, I'll prepare an example with Ignit= e 2.0 /=C2=A0ado.net=C2=A0and post it some time l= ater.

=C2=A0

++ User List

=C2=A0

any help much appreciated.

=C2=A0

Thanks And Regards

Chetan D

---------- Forwarded mes= sage ----------
From:=C2=A0= Pavel Tupitsyn=C2=A0<= ;ptupitsyn@apache.org>
Date: Fri, May 26, 2017= at 4:38 PM
Subject: Re: Write behind using Grid Gain
To: Chetan D &l= t;cchetands@gmail.com>

Hi Chetan, can you please write this to our user list,=C2=A0user@ignite.apache.or= g?

So that entire comm= unity can participate.

=C2=A0

Thanks,

Pavel

=C2=A0

On Fri, May 26, 2017 = at 1:35 PM, Chetan D <cchetands@gmail.com> wrot= e:

Hi Pavel Tupitsyn,

=C2=A0

= I have posted a comment in your blog as well (entity framework as ignite .n= et store) regarding write behind using ignite.

=C2=A0

I have been= working on a project where i need to implement distributed caching and i h= ave been asked to look into grid gain.

=C2=A0

This is is the firs= t time i am working on caching and this is entirely new topic for me.

=C2=A0

The example which you have shared i was able to understand a li= ttle and the sad part is even entity framework also i have never worked on.=

=C2=A0

It would be helpful if you can share me a simple example = using=C2=A0ado.net<= span class=3D"apple-converted-space">=C2=A0implementing read through= , write through and write behind even a simple table helps me understand th= e concept.

<= /div>
<= /div>

=C2=A0

--001a113d1a4242c1670551de8f56--