metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kasper Sørensen <i.am.kasper.soren...@gmail.com>
Subject Re: Adding more metadata (for instance about scale or other features) in MetaModel
Date Wed, 15 Jul 2015 10:41:07 GMT
That's absolutely true. It's not that I want to stop discovering what we
can, but I was more thinking of also adding a mechanism to plug your own
metadata. I guess it's pretty rare that a database itself offers a "domain
oriented" metadata system where I could tell it that "this field is a zip
code, and together with field X, Y and Z it forms a single address".

While from a querying perspective what I would love to archieve is that I
could express something like the following:

"Query all the addresses in the database".
"Do a SUM, AVG, MAX and MIN on all the ordinal-scale numbers in the
database".

It will also help a lot in generating templates for data integration. If I
am trying to move data from one table to another then I can probably do a
lot of automatic mapping based on the metadata. Same goes for reporting I
guess and stuff like that.

Best regards,
Kasper

2015-07-15 12:06 GMT+02:00 Alberto Rodriguez <ardlema@gmail.com>:

> Ok, so you are not thinking of discovering more metadata "on-the-fly", your
> approach is statically define metadata for the datasource and load and
> mix-in it with the existing metadata right?
>
> IMHO with this approach we will add a strong dependency between the data
> itself and the "external" metadata, correct me if I'm wrong or not fully
> understand your proposal but if the datastore changes (one column is
> deleted or a new column is added) the metadata will get obsolete.
>
> Kind regards,
>
> Alberto
>
> 2015-07-13 19:26 GMT+02:00 Kasper Sørensen <i.am.kasper.sorensen@gmail.com
> >:
>
> > I was thinking of having something like pluggable annotations or features
> > that could be added to tables, columns or groups of columns. Maybe also
> to
> > other entities. But since a lot of this is not available as a thing that
> > can be explored in the datastore itself I guess it would need to be
> stored
> > externally.
> >
> > Examples of features that I could imagine:
> >
> > Data type features: NominalScale, OrdinalScale, ...
> > Data conversion features: IntegerAsString, DateAsString, TimestampAsLong
> > ...
> > Domain features: FirstName, LastName, AddressLine, AddressCity,
> > AddressCountry, DateYear, DateMonth ...
> > And groupings of columns also in domain like fatures: Name (composed of
> > e.g. first and last name), Address (composed of multiple address fields),
> > Date (composed of year, month etc.)
> >
> > I would like to store them so that I can save and load them, saving the
> > developer for the work of restoring all the metadata again and again. Is
> > that not sensible?
> >
> > Best regards,
> > Kasper
> >
> >
> > 2015-07-13 9:55 GMT+02:00 Alberto Rodriguez <ardlema@gmail.com>:
> >
> > > Hi all,
> > >
> > > We are also facing similar issues so I completely agree with this
> > feature.
> > > In fact, we added recently in our service layer a new field for our
> > > metadata called "format", we needed this field to specify different
> > format
> > > types for the dates returned by our datastores.
> > >
> > > However, I'm not really sure how to implement this feature... I guess
> we
> > > should keep getting the metadata "core" from our datasources but what
> > about
> > > the new metadata??:
> > >
> > >    - How to fill it out? Will the integrator of MM provide functions to
> > > define when an element is going to be "x" and when is going to be "y"?
> > > (thinking here of providing lambda functions)
> > >    - Do we really need a metadata store?
> > >
> > > Kind regards,
> > >
> > >
> > > 2015-07-10 12:51 GMT+02:00 Kasper Sørensen <
> > i.am.kasper.sorensen@gmail.com
> > > >:
> > >
> > > > Hi all,
> > > >
> > > > All the time I see more and more need for us to add metadata to our
> > > > MetaModel based connectors. That could be for instance metadata about
> > > scale
> > > > (nominal, ordinal etc.) so that we can automate some stats collection
> > > etc.
> > > > or it could be more "meaning" oriented features to describe e.g.
> "This
> > > is a
> > > > first name" or "This is a city" or "These two fields (first and last
> > > name)
> > > > are together defining a name of a person".
> > > >
> > > > We have such mechanisms in our application levels many places, but
> not
> > at
> > > > the core framework of MetaModel and that's a pity because it makes it
> > > > harder for us to share.
> > > >
> > > > So I'm thinking of adding such a layer to the metadata of MetaModel.
> > But
> > > > one thing that's difficult is then about representing that metadata
> in
> > > some
> > > > "metadata store" which isn't necesarily the same as the data source
> > > itself.
> > > > It could be an XML file or it could be a complete metadata database.
> > And
> > > I
> > > > think that this metadata would be mutable by the integrator of
> > MetaModel
> > > > because it is rarely fully revealed by the data source itself.
> > > >
> > > > What do you think? Nice feature or?`
> > > >
> > > > Kasper
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message