Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <20151223080102.GA10680@tpx>
References: <20151222083212.GN10680@tpx>
 <CALH+G9qhXSKb-iBJfuD-W6i73yCDfB5eGZ66BNfVGb+RSXAUoQ@mail.gmail.com>
 <20151223033013.GT10680@tpx>
 <CA+0=VoUXBVR=c45GA5vrWTbZuh9VJ+0kvkNazZjRNu2avBschw@mail.gmail.com>
 <20151223080102.GA10680@tpx>
From: Dmitriy Setrakyan <dsetrakyan@apache.org>
Date: Wed, 23 Dec 2015 00:47:27 -0800
Message-ID: 
 <CA+0=VoUkb1cyJWeomV49TAu+Z=f5SeVXQ0Z2YBOXFHuitE8YMA@mail.gmail.com>
Subject: Re: Dynamic caches creation
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary=047d7bd6aaecf54b5b05278cc7fe

--047d7bd6aaecf54b5b05278cc7fe
Content-Type: text/plain; charset=UTF-8

Cos, I am confused. What is the behavior you would like to see?

On Wed, Dec 23, 2015 at 12:01 AM, Konstantin Boudnik <cos@apache.org> wrote:

> What if I don't know the configuration in advance? Doesn't it mean that I
> would have to restart the nodes whenever a new cache is configured and
> needs
> to be added to the cluster?
>
> Cos
>
> On Tue, Dec 22, 2015 at 10:37PM, Dmitriy Setrakyan wrote:
> > Cos,
> >
> > As far as schema-on-read, you can set all your caches in XML
> configuration,
> > and they will be pre-created for you. Will this do the trick?
> >
> > D.
> >
> > On Tue, Dec 22, 2015 at 7:30 PM, Konstantin Boudnik <cos@apache.org>
> wrote:
> >
> > > On Tue, Dec 22, 2015 at 03:45PM, Alexey Kuznetsov wrote:
> > > > Cos,
> > > >
> > > > How you are going to create caches in already started cluster?
> > > > I think you will create a cache configuration and after that will get
> > > cache
> > > > via Ignite.getOrCreateCache(ccfg).
> > >
> > > I would imagine that I can write some Java code to describe the
> > > configuration
> > > if needed. After all, Spring is all over the place. And while I am not
> a
> > > big
> > > fun of overusing it, it's already there, so perhaps it can do something
> > > useful ;)
> > >
> > > > So if you server cluster at some point was completely restarted, then
> > > > executing getOrCreateCache(ccfg) will create cache again (if needed)
> or
> > > > return existing cache.
> > >
> > > I am trying to have an analogy with either RDBMS or a data processing
> > > framework. I know the both of those aren't exact, but bear with me for
> a
> > > second. In RDBMS world the UX is to be able either to query existing
> > > tables or
> > > to create, populate and query new ones. No special auxiliary
> configurations
> > > are needed. In a data processing frameworks like Spark, Flink, etc. the
> > > data
> > > is originating the schema (aka schema on read) thus no special
> preparation
> > > steps is needed before the data could be read from a storage and
> processed.
> > >
> > > Now, in the case of Ignite the data needs to be transferred to a RAM,
> > > however
> > > same schema-on-read (a parsing code) or an externalized metadata
> (stored
> > > config or something) could be used to structure it on-the-fly. Hence,
> my
> > > cluster would have a higher level of runtime dynamic as I now I can
> create
> > > new
> > > caches as I go, without restarting the cluster nodes on every sneeze.
> > >
> > > > Also AFAIK CacheConfiguration class is serializable - you can save it
> > > > somewhere and later load if needed.
> > > > Or you may define some XML files with cache beans and load them with
> > > > IgniteSpringHelper.
> > > >
> > > > Thoughts?
> > >
> > > Basically, I am trying to think of how to make this whole thing more
> > > user-friendly and less middleware developers oriented. Wouldn't it be
> > > great if
> > > a user can load some external data and immediately start playing with
> it
> > > and
> > > doing some OLAP or even - oh horror :) - OLTP on it?
> > >
> > > Does it make sense?
> > >   Cos
> > >
> > > > On Tue, Dec 22, 2015 at 3:32 PM, Konstantin Boudnik <cos@apache.org>
> > > wrote:
> > > >
> > > > > Guys,
> > > > >
> > > > > is it possible to configure caches dynamically and persist their
> > > > > configuration
> > > > > in some shape and form? Here's the use case:
> > > > >  - we want to create some caches in the already running cluster
> for a
> > > data
> > > > > set
> > > > >  - once it is done, we'll run some SQL queries on top of those
> > > > >  - ideally, we'd like to be able to safe the cache configurations
> so
> > > next
> > > > >    time, we don't need to do the structures and field types
> discovery,
> > > but
> > > > >    instead be able to load it on (re)start
> > > > >
> > > > > Is this a supported use-case or everything should be defined
> statically
> > > > > before
> > > > > nodes start? Looks like the latter, but perhaps we are missing
> > > something.
> > > > >
> > > > > Thanks in advance for any info,
> > > > >   Cos
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Alexey Kuznetsov
> > > > GridGain Systems
> > > > www.gridgain.com
> > >
>

--047d7bd6aaecf54b5b05278cc7fe--