ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: GridGain Donates Persistent Distributed Store To ASF (Apache Ignite)
Date Wed, 19 Apr 2017 23:46:03 GMT
>> 5. Can we have a discussion about the design of this new layer so people
>> here
>>   can understand better what's being offered, how to take the advantage of
>>   it, and - most importantly - to offer their own insights and
>> improvements
>>   into this new subsystems before it's landed in the source code? And it
>>   would safe a lot of time on Q&A as well.
>> 
> 
> Yes, good idea. Denis, do you have a high level architecture for the
> proposed persistence?

It will be prepared right after Apache Ignite 2.0 release. Presently, don’t have time to
wrap it up in a doc form.

—
Denis

> On Apr 18, 2017, at 10:44 PM, Dmitriy Setrakyan <dsetrakyan@apache.org> wrote:
> 
> Cos, good questions! My answers are inline...
> 
> On Tue, Apr 18, 2017 at 4:17 PM, Konstantin Boudnik <cos@apache.org <mailto:cos@apache.org>>
wrote:
> 
>> Great news indeed! Thanks for sharing!
>> 
>> Before we jump on the voting and all that, can we have a chance to learn
>> more
>> about this new feature and its integration points with the rest of the
>> platform? A few questions come to mind immediately:
>> 
>> 1. This is an "optional disk layer", so it could be turned off
>> _completely_ and
>>   have no effect on those who don't want nor need to use it, right?
>> 
> 
> Yes
> 
> 
>> 2. Does it have any performance implications on the in-memory operations?
>> 
> 
> No, as long as the persistence is turned off, the in-memory operations will
> not be impacted.
> 
> 
>> 3. When you say it is "fully ... ANSI-99 SQL compliant fault-tolerant"
>> does it
>>   mean that _all_ SQL operations are now now supported through SQL or
>> still
>>   some of them only available through the JAVA APIs? THe fault tolerance
>> is
>>   for the data-center only as before, right? No new WAN-able HA has been
>>   introduced?
>> 
> 
> Well... I would say most SQL operations are going to be supported,
> including CREATE, DROP, INSERT, UPDATE, DELETE, SELECT, and of course,
> distributed joins.
> 
> And yes, you are right about the fault tolerance.
> 
> 
>> 4. With addition of this new model, are there any backward compatibility
>>   issues that would affect Ignite's application developers?
>> 
> 
> I don't think so. All incompatible changes should have been introduced in
> 2.0. I will let other community members comment here...
> 
> 
>> 5. Can we have a discussion about the design of this new layer so people
>> here
>>   can understand better what's being offered, how to take the advantage of
>>   it, and - most importantly - to offer their own insights and
>> improvements
>>   into this new subsystems before it's landed in the source code? And it
>>   would safe a lot of time on Q&A as well.
>> 
> 
> Yes, good idea. Denis, do you have a high level architecture for the
> proposed persistence?
> 
> 
>> 
>> I am confused a little bit by these two slightly controversial statements:
>> - "GG... has been developing a unique distributed persistent store...for
>> more
>>  than a year in-house"
>> - "we decided at GridGain that this tremendous feature should be open
>> source
>>  from the very beginning"
>> 
>> So, it sounds like the code has been under the development for a while and
>> it
>> isn't opened up "from the very beginning", unless there's a new meaning of
>> the
>> word beginning I am not aware of just yet :) It feels like this could be a
>> significant amount of the code to be digested by the community.
>> 
> 
> Yes, you are right. Many of us wanted to open source this functionality
> from the get go. In any case, this makes a great addition to the project. I
> hope we will be able to provide enough documentation and feedback on the
> dev list to ease up the digestion process.
> 
> 
>> 
>> Appreciate your thoughts on this! Thanks,
>>  Cos
>> 
>> On Mon, Apr 17, 2017 at 07:37PM, Denis Magda wrote:
>>> Igniters,
>>> 
>>> GridGain, as one of the most active Apache Ignite contributors, has been
>>> developing a unique distributed persistent store specifically for Apache
>>> Ignite for more than a year in-house. It’s a fully ACID and ANSI-99 SQL
>>> compliant fault-tolerant solution.
>>> 
>>> The store transparently integrates with Apache Ignite as an optional disk
>>> layer (in addition to the existing RAM layer) via the new page memory
>>> architecture that to be released in Apache Ignite 2.0. This allows
>> storing
>>> supersets of data on disk while having a subset in memory not worrying
>> about
>>> that you forgot to preload (warmup) your caches!
>>> 
>>> Assuming that the storage goes to ASF as a part of Apache Ignite 2.1
>> release
>>> the following will be supported by Ignite out-of-the-box:
>>> 
>>> * SQL queries execution over the data that is both in RAM and on disk: no
>>> need to preload the whole data set in memory.
>>> 
>>> * Cluster instantaneous restarts: once your cluster ring is recovered
>> after
>>> a restart your applications are fully operational even if they highly
>>> utilize SQL queries.
>>> 
>>> As for the use cases, it means that Apache Ignite will be possible to
>> use as
>>> a distributed SQL database with memory-first concept.
>>> 
>>> And we decided at GridGain that this tremendous feature should be open
>>> source from the very beginning.
>>> 
>>> Guys, could you advise how I can start official donation process?
>>> 
>>> —
>>> Denis


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message