ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Dynamic caches creation
Date Wed, 23 Dec 2015 06:37:39 GMT
Cos,

As far as schema-on-read, you can set all your caches in XML configuration,
and they will be pre-created for you. Will this do the trick?

D.

On Tue, Dec 22, 2015 at 7:30 PM, Konstantin Boudnik <cos@apache.org> wrote:

> On Tue, Dec 22, 2015 at 03:45PM, Alexey Kuznetsov wrote:
> > Cos,
> >
> > How you are going to create caches in already started cluster?
> > I think you will create a cache configuration and after that will get
> cache
> > via Ignite.getOrCreateCache(ccfg).
>
> I would imagine that I can write some Java code to describe the
> configuration
> if needed. After all, Spring is all over the place. And while I am not a
> big
> fun of overusing it, it's already there, so perhaps it can do something
> useful ;)
>
> > So if you server cluster at some point was completely restarted, then
> > executing getOrCreateCache(ccfg) will create cache again (if needed) or
> > return existing cache.
>
> I am trying to have an analogy with either RDBMS or a data processing
> framework. I know the both of those aren't exact, but bear with me for a
> second. In RDBMS world the UX is to be able either to query existing
> tables or
> to create, populate and query new ones. No special auxiliary configurations
> are needed. In a data processing frameworks like Spark, Flink, etc. the
> data
> is originating the schema (aka schema on read) thus no special preparation
> steps is needed before the data could be read from a storage and processed.
>
> Now, in the case of Ignite the data needs to be transferred to a RAM,
> however
> same schema-on-read (a parsing code) or an externalized metadata (stored
> config or something) could be used to structure it on-the-fly. Hence, my
> cluster would have a higher level of runtime dynamic as I now I can create
> new
> caches as I go, without restarting the cluster nodes on every sneeze.
>
> > Also AFAIK CacheConfiguration class is serializable - you can save it
> > somewhere and later load if needed.
> > Or you may define some XML files with cache beans and load them with
> > IgniteSpringHelper.
> >
> > Thoughts?
>
> Basically, I am trying to think of how to make this whole thing more
> user-friendly and less middleware developers oriented. Wouldn't it be
> great if
> a user can load some external data and immediately start playing with it
> and
> doing some OLAP or even - oh horror :) - OLTP on it?
>
> Does it make sense?
>   Cos
>
> > On Tue, Dec 22, 2015 at 3:32 PM, Konstantin Boudnik <cos@apache.org>
> wrote:
> >
> > > Guys,
> > >
> > > is it possible to configure caches dynamically and persist their
> > > configuration
> > > in some shape and form? Here's the use case:
> > >  - we want to create some caches in the already running cluster for a
> data
> > > set
> > >  - once it is done, we'll run some SQL queries on top of those
> > >  - ideally, we'd like to be able to safe the cache configurations so
> next
> > >    time, we don't need to do the structures and field types discovery,
> but
> > >    instead be able to load it on (re)start
> > >
> > > Is this a supported use-case or everything should be defined statically
> > > before
> > > nodes start? Looks like the latter, but perhaps we are missing
> something.
> > >
> > > Thanks in advance for any info,
> > >   Cos
> > >
> >
> >
> >
> > --
> > Alexey Kuznetsov
> > GridGain Systems
> > www.gridgain.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message