ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Schema in CacheConfig is not updated after DDL commands(Add/drop column, Create/drop index)
Date Wed, 12 Dec 2018 08:16:49 GMT
Hi Nikolay,

I think yes.

On Mon, Dec 10, 2018 at 9:10 PM Nikolay Izhikov <nizhikov@apache.org> wrote:

> Vladimir.
>
> Since we are talking about internal component(Spark Data Frame integration)
> can we use same technique as JdbcRequestHandler to obtain columns metadata?
> Seems, code above will fetch up to date columns info:
>
>
> org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler#getColumnsMeta
> ```
>
>             for (String cacheName : ctx.cache().publicCacheNames()) {
>                 for (GridQueryTypeDescriptor table :
> ctx.query().types(cacheName)) {
>                     if (!matches(table.schemaName(), req.schemaName()))
>                         continue;
>
>                     if (!matches(table.tableName(), req.tableName()))
>                         continue;
>         //Process column here
>
> ```
>
> пн, 19 нояб. 2018 г. в 18:54, Vladimir Ozerov <vozerov@gridgain.com>:
>
> > There is no public API currently. This information can be extracted using
> > JDBC metadata requests.
> >
> > On Mon, Nov 19, 2018 at 6:50 PM Valentin Kulichenko <
> > valentin.kulichenko@gmail.com> wrote:
> >
> > > Vladimir,
> > >
> > > What is the best way to get current schema information (list of tables,
> > > columns, etc.)?
> > >
> > > -Val
> > >
> > > On Mon, Nov 19, 2018 at 1:21 AM Vladimir Ozerov <vozerov@gridgain.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > In this case Spark integration should be fixed. as we never stated
> that
> > > DDL
> > > > updates will be reflected in IgniteCache.getConfiguration().
> > > >
> > > >
> > > > On Mon, Nov 19, 2018 at 11:58 AM Ray <rayliu@cisco.com> wrote:
> > > >
> > > > > When user performs column and index modification operation in
> SQL(ex
> > > > create
> > > > > index, drop index, add column, drop column),  QueryEntity in
> > > > > CacheConfiguration for the modified cache is not updated.
> > > > >
> > > > > Here's my analysis
> > > > >
> > > > > QueryEntity in QuerySchema is a local copy of the original
> > QueryEntity,
> > > > so
> > > > > the original QueryEntity is not updated when modification happens.
> > > > >
> > > > > I have created a ticket for this issue
> > > > > https://issues.apache.org/jira/browse/IGNITE-10314
> > > > >
> > > > > But as Vlad said in the comments "public configuration is
> immutable,
> > it
> > > > > represents initial cache parameters. So it is expected that
> > > configuration
> > > > > will not be updated after DDL commands. Real changes are
> accumulated
> > in
> > > > > separate query entity which is hidden from user and used
> internally"
> > > > >
> > > > > But I think it's only reasonable to return the newest QueryEntity
> to
> > > > user.
> > > > >
> > > > > For example, a user adds a column to a table then he reads data
> using
> > > > Spark
> > > > > data frame API which currently relies on QueryEntity to construct
> > data
> > > > > frame
> > > > > schema, so user will get wrong schema.
> > > > >
> > > > > What do you guys think?
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message