Return-Path: X-Original-To: apmail-ignite-dev-archive@minotaur.apache.org Delivered-To: apmail-ignite-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 11DAB18B11 for ; Thu, 22 Oct 2015 12:01:45 +0000 (UTC) Received: (qmail 9817 invoked by uid 500); 22 Oct 2015 12:01:45 -0000 Delivered-To: apmail-ignite-dev-archive@ignite.apache.org Received: (qmail 9774 invoked by uid 500); 22 Oct 2015 12:01:44 -0000 Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ignite.apache.org Delivered-To: mailing list dev@ignite.apache.org Received: (qmail 9763 invoked by uid 99); 22 Oct 2015 12:01:44 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Oct 2015 12:01:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 43CD2180A38 for ; Thu, 22 Oct 2015 12:01:44 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.98 X-Spam-Level: ** X-Spam-Status: No, score=2.98 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id fh_gSN8IWv7f for ; Thu, 22 Oct 2015 12:01:37 +0000 (UTC) Received: from mail-yk0-f182.google.com (mail-yk0-f182.google.com [209.85.160.182]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 72E9E20B89 for ; Thu, 22 Oct 2015 12:01:36 +0000 (UTC) Received: by ykba4 with SMTP id a4so73502402ykb.3 for ; Thu, 22 Oct 2015 05:01:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=BoRYzKV4CQ/Rja1zu0OEUJkMYoEkEGptHgMoGzS4Ez8=; b=f6kNHFyONMKeKt+8eJXN+dnzKLn98LpJRTXxqn+xHV4dauOzq5i3u9edpPMrm9uBA5 Y60WFZuN2mbGFooGx5K5nwano9PgE+TDp0sh+a4oEOZ9BFvOtsotEg8aQUQyzVJx0WOw c19mpLGjmMemdH5NyvQCCB70uM0SwBeCZill49cDeV/o1yFWGv8es8VUV6BnoHlRrLwK Rp1UAOf/cT5jy54gJD0kj4bEsUnJTtHLwiPVqrSXb4F/2Z8dKBZBmeIIR+bUA9ziTHHf L79Awg2PYw7Nqe+KNKocAOs+i2lX6rEdPeYM7Lo7JwWc68h4jAm7pQI85jwG1Ny0bcR2 8ykw== X-Gm-Message-State: ALoCoQn3mH9R6wyma0w4atAqMy7QqM/ZYs28BcR9lOvCCuABXIOqtUMLtbzD76jBPAXDPmPSxi8/ MIME-Version: 1.0 X-Received: by 10.13.223.17 with SMTP id i17mr11639131ywe.189.1445515295280; Thu, 22 Oct 2015 05:01:35 -0700 (PDT) Received: by 10.13.223.84 with HTTP; Thu, 22 Oct 2015 05:01:35 -0700 (PDT) In-Reply-To: References: <20151021234813.GI10001@tpx> <20151022014818.GJ10001@tpx> Date: Thu, 22 Oct 2015 19:01:35 +0700 Message-ID: Subject: Re: Data Snapshots in Ignite From: Alexey Kuznetsov To: dev@ignite.apache.org Content-Type: multipart/alternative; boundary=001a114e4798b1b1f80522b04149 --001a114e4798b1b1f80522b04149 Content-Type: text/plain; charset=UTF-8 >> Not sure how CQEngine will help for snapshotting of Ignite. Can you explain? There was a discussion about this lib has various types of indexes that could be only created on read-only data. But it seems that indeed my note has a little value for this thread. On Thu, Oct 22, 2015 at 2:31 PM, Dmitriy Setrakyan wrote: > Not sure how CQEngine will help for snapshotting of Ignite. Can you > explain? > > On Wed, Oct 21, 2015 at 7:48 PM, Alexey Kuznetsov > > wrote: > > > Some time ago we already discussed snapshots idea while discussing > library > > "cqengine" [1]. > > > > May be this library could be useful in snapshot mode. > > > > [1] Sources: https://github.com/npgall/cqengine > > Docs: https://code.google.com/p/cqengine/ > > > > On Thu, Oct 22, 2015 at 8:48 AM, Konstantin Boudnik > > wrote: > > > > > On Wed, Oct 21, 2015 at 05:58PM, Dmitriy Setrakyan wrote: > > > > On Wed, Oct 21, 2015 at 4:48 PM, Konstantin Boudnik > > > wrote: > > > > > > > > > I like it quite a bit, as well! Ticket would make the most sense as > > > well, > > > > > so > > > > > there will be a single place to collect the design docs (if > needed), > > > etc. > > > > > > > > > > On Wed, Oct 21, 2015 at 04:45PM, Dmitriy Setrakyan wrote: > > > > > > I also really like the idea. One potential use case is fraud > > > analysis in > > > > > > financial institutions. Rarely it makes sense to perform such > > > analysis > > > > > on a > > > > > > life system, but rather a snapshot of some data needs to be taken > > and > > > > > > analyzed offline. > > > > > > > > > > > > I think snapshots should be saved to disk, so users could load > them > > > for > > > > > > analysis on a totally different cluster. > > > > > > > > > > I think disk persistence should be optional, not mandatory. > > > > > > > > > > > > > I would actually prefer to support disk-only snapshots. I think it > will > > > be > > > > difficult (double-the-work) to support both, in-memory and disk > > formats. > > > > Also, storing snapshots in-memory would require extra memory (a lot > of > > > it) > > > > for something that gets saved mainly for historic purposes or offline > > > > analysis. > > > > > > Ah, good points! Agree. > > > > > > > > > > > > -- > > Alexey Kuznetsov > > GridGain Systems > > www.gridgain.com > > > -- Alexey Kuznetsov GridGain Systems www.gridgain.com --001a114e4798b1b1f80522b04149--