ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Write behind using Grid Gain
Date Tue, 13 Jun 2017 21:40:55 GMT
Raymond,

Then Ignite Persistent Store is exactly for your use case. Please refer to this discussion
on the dev list:
http://apache-ignite-developers.2346864.n4.nabble.com/GridGain-Donates-Persistent-Distributed-Store-To-ASF-Apache-Ignite-td16788.html#a16838

Also it was covered a bit in that webinar:
https://www.youtube.com/watch?v=bDrGueQ16UQ

The store should be released by the community in the nearest.

—
Denis

> On Jun 13, 2017, at 2:02 PM, Raymond Wilson <raymond_wilson@trimble.com> wrote:
> 
> Hi Pavel,
>  
> It’s a little complicated. The system is essentially a DB in its own right; actually
it’s an IMDG a bit like Ignite, but developed 8 years ago to fulfill a need we had. 
>  
> Today, I am looking to modernize that system and rather than continuing to build and
maintain all the core ‘infrastructure’ features of an IMDG such as clustering, messaging,
enterprise caching etc, I am looking to see how well Ignite fits by running a Proof of Concept
project. It turns out it fits quite well, largely because the architectural structure of both
systems (ie: IMDG) is well aligned in terms of the problems being solved.
>  
> The primary gap between the legacy system and IMDG is that IMDG does not support persistence.
The legacy system has a distributed cache that stores objects that are aggregate collections
(10’s of thousand’s) of relatively simple spatial data records that are operated on by
the clustered compute engine. Sometimes billions of records need to be processed to satisfy
a single query. Your standard run of the mill SQL DB finds these sorts of queries hard.
>  
> I suppose you could use another DB (MS-SQL, AWS:RDS etc) to store those aggregate blobs,
but it seems like a bit of a ‘miss-use case’ when what I’m really after is a persistence/storage
layer J
>  
> Thanks,
> Raymond.
>  
>  
> From: Pavel Tupitsyn [mailto:ptupitsyn@apache.org <mailto:ptupitsyn@apache.org>]

> Sent: Wednesday, June 14, 2017 1:59 AM
> To: user@ignite.apache.org <mailto:user@ignite.apache.org>
> Subject: Re: Write behind using Grid Gain
>  
> Hi Raymond,
>  
> I think your use case fits well into traditional Ignite model of write-through cache
store with backing database.
> Why do you want to avoid a DB? Do you plan to store data on disk directly as a set of
files?
>  
> Pavel
>  
> On Mon, Jun 12, 2017 at 2:14 AM, Raymond Wilson <raymond_wilson@trimble.com <mailto:raymond_wilson@trimble.com>>
wrote:
> Hi Pavel,
>  
> Thanks for the blog – it explains it quite well.
>  
> I have a slightly different use case where groups of records within a much larger data
set are clustered together for efficiency (ie: each of the cached items in the Ignite grid
cache has significant internal structure). You can think of them as a large number of smallish
files (a few Kb to a few Mb), but file systems don’t like lots of small files. 
>  
> I have a legacy implementation that houses these small files within a single larger file,
but wanted to know if there was a clean way of supporting the same structure using the Ignite
read/write through support, perhaps with another system providing relatively transparent persistency
semantics but which does not use a DB to store the data.
>  
> Thanks,
> Raymond.
>  
> From: Pavel Tupitsyn [mailto:ptupitsyn@apache.org <mailto:ptupitsyn@apache.org>]

> Sent: Saturday, May 27, 2017 5:03 AM
> To: user@ignite.apache.org <mailto:user@ignite.apache.org>
> 
> Subject: Re: Write behind using Grid Gain
>  
> I've decided to write a blog post, since this topic seems to be in demand:
> https://ptupitsyn.github.io/Ado-Net-Cache-Store/ <https://ptupitsyn.github.io/Ado-Net-Cache-Store/>
>  
> Code:
> https://github.com/ptupitsyn/ignite-net-examples/tree/master/AdoNetCacheStore <https://github.com/ptupitsyn/ignite-net-examples/tree/master/AdoNetCacheStore>
>  
> Let me know if this helps!
>  
> On Fri, May 26, 2017 at 3:50 PM, Chetan D <cchetands@gmail.com <mailto:cchetands@gmail.com>>
wrote:
> Thank you Pavel.
>  
> waiting for your response.
>  
> On Fri, May 26, 2017 at 6:03 PM, Pavel Tupitsyn <ptupitsyn@apache.org <mailto:ptupitsyn@apache.org>>
wrote:
> To give everyone context, this is not about GridGain, but about Apache Ignite.
> The blog post in question is https://ptupitsyn.github.io/Entity-Framework-Cache-Store/
<https://ptupitsyn.github.io/Entity-Framework-Cache-Store/>
>  
> Chetan, I'll prepare an example with Ignite 2.0 / ado.net <http://ado.net/> and
post it some time later.
>  
> Pavel
>  
> On Fri, May 26, 2017 at 2:32 PM, Chetan D <cchetands@gmail.com <mailto:cchetands@gmail.com>>
wrote:
> ++ User List
>  
> any help much appreciated.
>  
> Thanks And Regards
> Chetan D
> ---------- Forwarded message ----------
> From: Pavel Tupitsyn <ptupitsyn@apache.org <mailto:ptupitsyn@apache.org>>
> Date: Fri, May 26, 2017 at 4:38 PM
> Subject: Re: Write behind using Grid Gain
> To: Chetan D <cchetands@gmail.com <mailto:cchetands@gmail.com>>
> 
> Hi Chetan, can you please write this to our user list, user@ignite.apache.org <mailto:user@ignite.apache.org>?
> So that entire community can participate.
>  
> Thanks,
> Pavel
>  
> On Fri, May 26, 2017 at 1:35 PM, Chetan D <cchetands@gmail.com <mailto:cchetands@gmail.com>>
wrote:
> Hi Pavel Tupitsyn,
>  
> I have posted a comment in your blog as well (entity framework as ignite .net store)
regarding write behind using ignite.
>  
> I have been working on a project where i need to implement distributed caching and i
have been asked to look into grid gain.
>  
> This is is the first time i am working on caching and this is entirely new topic for
me.
>  
> The example which you have shared i was able to understand a little and the sad part
is even entity framework also i have never worked on.
>  
> It would be helpful if you can share me a simple example using ado.net <http://ado.net/>
implementing read through, write through and write behind even a simple table helps me understand
the concept.


Mime
View raw message