Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAKoNANurc_W4SvEwcAq5R33PjfOwPpsLkyR_gZbSi+fzBCQVBg@mail.gmail.com>
References: 
 <CAKoNANuGucJnksGJQtjkX+8RyAJ6M8NJc16e_+=xvnq6x1RPHA@mail.gmail.com>
 <CAKoNANurc_W4SvEwcAq5R33PjfOwPpsLkyR_gZbSi+fzBCQVBg@mail.gmail.com>
From: Dmitriy Setrakyan <dsetrakyan@apache.org>
Date: Mon, 26 Oct 2015 18:53:50 -0700
Message-ID: 
 <CA+0=VoW7j1pspy2gabEeQ_PS2qkShhmJcQ4oXYKtFz0S5D_Ssw@mail.gmail.com>
Subject: Re: Data Snapshots in Ignite
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary=e89a8fb200b4c540d105230c5b05

--e89a8fb200b4c540d105230c5b05
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Mon, Oct 26, 2015 at 5:31 AM, Raul Kripalani <raulk@apache.org> wrote:

> Hi,
>
> Thanks all for chiming in. It seems like this feature could be of interes=
t
> to the user community, so I've opened a ticket to continue maturing the
> idea there:
>
> https://issues.apache.org/jira/browse/IGNITE-1789
>
> We may need to create a Wiki page later to collaborate around specifics a=
nd
> design.
>

Thanks Raul. I agree that Wiki page may be in order. I have responded in
the ticket. Take a look and see if you agree with my thinking.


> Regards,
>
> *Ra=C3=BAl Kripalani*
> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
> Messaging Engineer
> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
> http://blog.raulkr.net | twitter: @raulvk
>
> On Wed, Oct 21, 2015 at 10:06 AM, Raul Kripalani <raulk@apache.org> wrote=
:
>
> > Hey guys,
> >
> > LevelDb has a functionality called Snapshots which provides a consisten=
t
> > read-only view of the DB at a given point in time, against which querie=
s
> > can be executed.
> >
> > To my knowledge, this functionality doesn't exist in the world of open
> > source In-Memory Computing. Ignite could be an innovator here.
> >
> > Ignite Snapshots would allow queries, distributed closures, map-reduce
> > jobs, etc. It could be useful for Spark RDDs to avoid data shift while
> the
> > computation is taking place (not sure if there's already some form of
> > snapshotting, though). Same for IGFS.
> >
> > Example usage:
> >
> >     IgniteCacheSnapshot snapshot =3D
> > ignite.cache("mycache").snapshots().create();
> >
> >     // all three queries are executed against a view of the cache at th=
e
> > point in time where it was snapshotted
> >     snapshot.query("select ...");
> >     snapshot.query("select ...");
> >     snapshot.query("select ...");
> >
> > In fact, it would be awesome to be able to logically save this snapshot
> > with a name so that later jobs, queries, etc. can run on top of it, e.g=
.:
> >
> >     IgniteCacheSnapshot snapshot =3D
> > ignite.cache("mycache").snapshots().create("abc");
> >
> >     // ...
> >     // in another module of a distributed system, or in another thread =
in
> > parallel, use the saved snapshot
> >     IgniteCacheSnapshot snapshot =3D
> > ignite.cache("mycache").snapshots().get("abc");
> >     ....
> >
> > Named snapshotting can be dangerous due to data retention, e.g. imagine
> > keeping a snapshot for 2 weeks! So we should force the user to specify =
a
> > TTL:
> >
> >     IgniteCacheSnapshot snapshot =3D
> > ignite.cache("mycache").snapshots().create("abc", 2, TimeUnit.HOURS);
> >
> > Such functionality would allow for "reporting checkpoints" and "time
> > travel", for example, where you want users to be able to query the data
> as
> > it stood 1 hour ago, 2 hours ago, etc.
> >
> > What do you think?
> >
> > P.S.: We do have some form of snapshotting in the Compute checkpointing
> > functionality =E2=80=93 but my proposal is to generalise the notion.
> >
> > Regards,
> >
> > *Ra=C3=BAl Kripalani*
> > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data a=
nd
> > Messaging Engineer
> > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalan=
i
> > http://blog.raulkr.net | twitter: @raulvk
> >
>

--e89a8fb200b4c540d105230c5b05--