metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kasper Sørensen <i.am.kasper.soren...@gmail.com>
Subject Re: [DISCUSS] Documentation improvements?
Date Thu, 22 Jan 2015 17:57:03 GMT
Nice. Thanks Tomasz. Now I guess we should work on how to bring some of
these new documentation items to the website. Any ideas?

2015-01-21 16:47 GMT+01:00 Tomasz Guziałek <
Tomasz.Guzialek@humaninference.com>:

> A new page in the Wiki is now online:
> https://wiki.apache.org/metamodel/GettingStartedNewConnectors
>
> Best regards,
> Tomasz
>
> -----Original Message-----
> From: Tomasz Guziałek [mailto:Tomasz.Guzialek@HumanInference.com]
> Sent: Wednesday, January 21, 2015 12:47
> To: dev@metamodel.apache.org
> Subject: RE: [DISCUSS] Documentation improvements?
>
> Thanks, it is working now :).
>
> Best regards,
> Tomasz
>
> -----Original Message-----
> From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
> Sent: Wednesday, January 21, 2015 12:43
> To: dev@metamodel.apache.org
> Subject: Re: [DISCUSS] Documentation improvements?
>
> Figure it out :-) You have access now.
>
> 2015-01-21 12:38 GMT+01:00 Kasper Sørensen <i.am.kasper.sorensen@gmail.com
> >:
>
> > I've been granted admin rights now ... trying to figure out how to
> > give you access to write new pages Tomasz ... :-)
> >
> > 2015-01-20 16:47 GMT+01:00 Tomasz Guziałek <
> > Tomasz.Guzialek@humaninference.com>:
> >
> >> I forgot about it actually. However, my INFRA ticket is still
> unresolved:
> >> https://issues.apache.org/jira/browse/INFRA-8902 Hopefully, some of
> >> you can add me as a contributor to the Wiki when you get your admin
> rights back.
> >>
> >> Best regards,
> >> Tomasz
> >>
> >> -----Original Message-----
> >> From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
> >> Sent: Wednesday, January 14, 2015 23:22
> >> To: dev@metamodel.apache.org
> >> Subject: Re: [DISCUSS] Documentation improvements?
> >>
> >> Hi everybody,
> >>
> >> I know it's been a while, but I felt like bumping this topic. I was
> >> thinking a bit about "what's most important for MetaModel at this point"
> >> and I think documentation is high on my list, if not highest. If the
> >> wiki is in any way blocking work (Tomasz?) then please raise it here
> >> and let's get it fixed. And while waiting for such a thing we should
> >> still be able to work on the docs - either via emails, documentation
> >> in .md files or whatever.
> >>
> >> Cheers,
> >> Kasper
> >>
> >> 2014-12-18 21:20 GMT+01:00 Kasper Sørensen <
> >> i.am.kasper.sorensen@gmail.com>:
> >>
> >> > Javadocs are now available at:
> >> >
> >> > http://metamodel.apache.org/apidocs/3.4.1
> >> >
> >> > or via this redirect url that we can always make point to new
> versions:
> >> > http://metamodel.apache.org/apidocs/current
> >> >
> >> > Kasper
> >> >
> >> >
> >> > 2014-12-17 17:02 GMT+01:00 Henry Saputra <henry.saputra@gmail.com>:
> >> >>
> >> >> Hi Tomasz,
> >> >>
> >> >> Hmm somehow I also do not have admin access to add individual to
> >> >> edit access, unfortunately you need to file ASF INFRA ticket to
> >> >> get help on this =(
> >> >>
> >> >> - Henry
> >> >>
> >> >> On Wed, Dec 17, 2014 at 3:33 AM, Tomasz Guziałek
> >> >> <Tomasz.Guzialek@humaninference.com> wrote:
> >> >> > I registered myself at the Wiki, but I believe I do not have the
> >> >> > rights
> >> >> for editing. Henry, is that you who can fix that for me? My
> >> >> username is "tomaszguzialek". Thank you!
> >> >> >
> >> >> > Best regards,
> >> >> > Tomasz
> >> >> >
> >> >> > -----Original Message-----
> >> >> > From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
> >> >> > Sent: Tuesday, December 16, 2014 20:04
> >> >> > To: dev@metamodel.apache.org
> >> >> > Subject: Re: [DISCUSS] Documentation improvements?
> >> >> >
> >> >> > Hi Tomasz,
> >> >> >
> >> >> > Good info too! I like the analogies to existing implementations
> >> >> > - I can
> >> >> imagine these are valueable examples for implementors.
> >> >> >
> >> >> > Maybe you could start a page on the MM wiki about this? To begin
> >> >> > with
> >> >> it could even be just a dump of that email. We can adjust and
> >> >> refine it easily when it's in that wiki format I think.
> >> >> >
> >> >> > I would call this page something like
> "GettingStartedNewConnectors".
> >> >> >
> >> >> > Best regards,
> >> >> > Kasper
> >> >> >
> >> >> > 2014-12-15 12:48 GMT+01:00 Tomasz Guziałek <
> >> >> > Tomasz.Guzialek@humaninference.com>:
> >> >> >>
> >> >> >> Hello Alberto and others,
> >> >> >>
> >> >> >> My first thought also was around building new connectors -
what
> >> >> >> is the minimum I need to provide as a developer to query a
new
> >> >> >> datastore and what I will get "for free" from MetaModel. My
> >> >> >> (quick sketch of) step-by-step guide would look like this:
> >> >> >>
> >> >> >> 1) Create a new XXXDataContext class that extends
> >> >> >> QueryPostProcessDataContext (to get functionalities for free)
> >> >> >> that holds a reference to a native API object. Taking existing
> examples:
> >> >> >>         CouchDbDataContext holds Ektorp' CouchDbInstance
> >> >> >>         HBaseDataContext holds HTablePool
> >> >> >>         CsvDataContext holds a handle to a file (through
> >> >> >> MetaModel's Resource class, to be precise)
> >> >> >> 2) Implement materializeMainSchemaTable to fetch the data
that
> >> >> >> is going to represent the table, using the native API. Existing
> >> examples:
> >> >> >>         CouchDbDataContext fetches a view will all docs through
> >> >> >> Ektorp's CouchDbInstance
> >> >> >>         HBaseDataContext creates a Scan of a table through
> >> >> HBaseTablePool
> >> >> >>         CsvDataContext reads the whole file using our file
> >> >> >> handle
> >> >> >> 3) Return a new XXXDataSet instance with the native result
> >> >> >> passed as a parameter. XXXDataSet class takes the native result
> >> >> >> object and translates it to the MetaModel's Row objects.
> >> >> >>         CouchDbDataSet will parse the JSON document and
> >> >> >> instantiate MetaModel's Row object with this data
> >> >> >>         HBaseDataSet extracts the values from the Scan and
> >> >> >> instantiates MetaModel's Row object with the values
> >> >> >>         CsvDataSet translates a line from a CSV file into
> >> >> >> MetaModel's Row object
> >> >> >>
> >> >> >> This is the minimum that needs to be implemented. While we
have
> >> >> >> the native result translated to MetaModel's Row objects,
> >> >> >> selecting specific columns, filtering and so on we get for
free
> >> >> >> from QueryPostProcessDataContext class that we subclassed.
Of
> >> >> >> course, it is not the most performant way of querying, for
> >> >> >> example doing a full scan to get a single row by its primary
> >> >> >> key is optimized in
> >> many databases.
> >> >> >> Delegating primary key lookups, count queries, queries with
> >> >> >> simple WHERE clauses to the native API instead of
> >> >> >> post-processing it in Java is the next step for a developer
of a
> new MetaModel connector.
> >> >> >>
> >> >> >> I would benefit a lot from such a guide at the beginning of
my
> >> >> >> journey with MetaModel...
> >> >> >>
> >> >> >> Best regards,
> >> >> >> Tomasz
> >> >> >>
> >> >> >>
> >> >> >> -----Original Message-----
> >> >> >> From: Alberto Rodriguez [mailto:ardlema@gmail.com]
> >> >> >> Sent: Friday, December 12, 2014 11:42
> >> >> >> To: dev@metamodel.apache.org
> >> >> >> Cc: juan.rodriguez.hortala@gmail.com
> >> >> >> Subject: Re: [DISCUSS] Documentation improvements?
> >> >> >>
> >> >> >> Hi all,
> >> >> >>
> >> >> >> When I first came across the project I wasn't sure how and
> >> >> >> where MM
> >> >> "plays"
> >> >> >> with the data. I mean, for the different datasources I didn't
> >> >> >> know whether MM was fetching the data in memory and then
> >> >> >> performing joins, selects and so on or was delegating this
> >> >> >> behaviour to the native drivers. The table that you guys put
> together in the wiki:
> >> >> >> QueryExecutionStrategies <
> >> >> >> https://wiki.apache.org/metamodel/QueryExecutionStrategies>
is
> >> >> >> quite
> >> >> clarifying.
> >> >> >>
> >> >> >> In my particular case, as we needed new connectors that wasn't
> >> >> >> implemented yet and decided to contribute to the project
> >> >> >> implementing them, I was "fighting" with the code from day
0
> >> >> >> and I must say that the code is pretty self-explanatory. Kasper
> >> >> >> has been done a great job and if you have some Java experience
> >> >> >> you might have a new connector ready-to-use within few days.
I
> >> >> >> think we should make an effort and try to share this with
the
> >> >> >> world, MM is great as it is now but I think one of its main
> >> >> >> powerful features is that you can create your own connectors
> >> >> >> without much effort. It might be a good idea to create a
> >> >> >> section on the website like "Create your own connectors" and
a
> >> >> >> guide
> >> >> to implement them. What do you think?
> >> >> >>
> >> >> >> I would also be more than happy to help in any decision we
take
> >> >> >> regarding the documentation.
> >> >> >>
> >> >> >> Kind regards,
> >> >> >>
> >> >> >> 2014-12-12 9:32 GMT+01:00 Kasper Sørensen
> >> >> >> <i.am.kasper.sorensen@gmail.com
> >> >> >> >:
> >> >> >> >
> >> >> >> > Hi Juan,
> >> >> >> >
> >> >> >> > Thanks for that feedback. I think it helps a lot to see
what
> >> >> >> > relevant and good information we already have (some of
it on
> >> >> >> > third party sites
> >> >> >> > - we should look to integrate it some more) and also
what we
> >> >> >> > should improve (the javadoc thing is such a simple thing
-
> >> >> >> > I'll create a JIRA issue about doing it ASAP).
> >> >> >> >
> >> >> >> > Kasper
> >> >> >> >
> >> >> >> > 2014-12-11 20:36 GMT+01:00 Juan Rodríguez Hortalá <
> >> >> >> > juan.rodriguez.hortala@gmail.com>:
> >> >> >> > >
> >> >> >> > > Hi all,
> >> >> >> > >
> >> >> >> > > I don't have a lot of experience with Metamodel,
I did my
> >> >> >> > > first
> >> >> >> > experiments
> >> >> >> > > with it in the capstone project for a MSc, and then
I used
> >> >> >> > > it in my work combined with Apache Sqoop to import
data
> >> >> >> > > from a relational database,
> >> >> >> > where
> >> >> >> > > Metamodel was in charge of obtaining the data schema.
But
> >> >> >> > > maybe my
> >> >> >> > limited
> >> >> >> > > experience could be of use for answering this question.
> >> >> >> > > I think I discovered Metamodel when searching the
internet
> >> >> >> > > for a way to connect a BI visualization tool with
HBase. As
> >> >> >> > > the starting points I
> >> >> >> > used:
> >> >> >> > >
> >> >> >> > >  * This post
> >> >> >> > > http://www.infoq.com/articles/apache-incubator-metamodel
by
> >> >> >> > > Kasper, which I think is a pretty good quick start,
with
> >> >> >> > > several
> >> >> code snippets.
> >> >> >> > >  * This other post
> >> >> >> > > http://kasper.eobjects.org/2013/06/introducing-apache-metam
> >> >> >> > > ode l.ht ml by Kasper, which is more about concepts
and the
> >> >> >> > > origin of Metamodel.
> >> >> >> > >  * The code examples from the site
> >> >> >> > > http://metamodel.apache.org/#examples
> >> >> >> > > also help.
> >> >> >> > >  * I also generated the Javadoc documentation, and
put it
> >> >> >> > > in my github
> >> >> >> > for
> >> >> >> > > easy access
> >> >> >> > > http://juanrh.github.io/doc/ApacheMetamodel/apidocs/
> >> >> >> > > (I hope that is ok for you, by the way!).
> >> >> >> > >
> >> >> >> > > As I said, I haven't done anything much sophisticated,
but
> >> >> >> > > I do think the javadoc documentation and API structure
is
> >> >> >> > > pretty intuitive. What I miss most is documentation
about
> >> >> >> > > the design and architecture so I can
> >> >> >> > understand
> >> >> >> > > the implications in performance of using Metamodel
in
> >> >> >> > > different ways. I think that is important because
that
> >> >> >> > > could determine whether Metamodel could be applied
or not
> >> >> >> > > for a particular problem. But these are just my
personal
> >> >> >> > > subjective
> >> impressions.
> >> >> >> > >
> >> >> >> > > TL;DR: In my opinion the blog post
> >> >> >> > > http://www.infoq.com/articles/apache-incubator-metamodel
is
> >> >> >> > > a good candidate for the "getting started" guide
for
> Metamodel.
> >> >> >> > > Also an updated version of the javadoc documentation
hosted
> >> >> >> > > in the site would
> >> >> >> be nice.
> >> >> >> > >
> >> >> >> > > Greetings,
> >> >> >> > >
> >> >> >> > > Juan
> >> >> >> > >
> >> >> >> > >
> >> >> >> > > 2014-12-11 18:47 GMT+01:00 Kasper Sørensen <
> >> >> >> > i.am.kasper.sorensen@gmail.com
> >> >> >> > > >:
> >> >> >> > >
> >> >> >> > > > Hi all,
> >> >> >> > > >
> >> >> >> > > > Discussed with a colleague the other day the
perception
> >> >> >> > > > of MetaModel
> >> >> >> > when
> >> >> >> > > > coming to the project as a newcomer (new in
our company
> >> >> >> > > > and approaching
> >> >> >> > > the
> >> >> >> > > > community). He mentioned that the documentation
was too
> >> >> >> > > > sparse for him
> >> >> >> > to
> >> >> >> > > > get a good insight by just browsing our website
and wiki.
> >> >> >> > > > A proper
> >> >> >> > > getting
> >> >> >> > > > started and better reference documentation
would have
> >> >> >> > > > helped him a lot
> >> >> >> > -
> >> >> >> > > > and improved his overall impression (which
has improved
> >> >> >> > > > as he started
> >> >> >> > > using
> >> >> >> > > > it, but wasn't immediately too good).
> >> >> >> > > >
> >> >> >> > > > I think that's pretty good feedback - documentation
is
> >> >> >> > > > important and we have not spent a lot of time
on it I guess.
> >> >> >> > > >
> >> >> >> > > > So are there any "low hanging fruit" we can
start by
> grabbing?
> >> >> >> > > > And what should be the more dedicated efforts
that we go
> >> >> >> > > > for
> >> >> here?
> >> >> >> > > >
> >> >> >> > > > I was thinking of at least publishing our javadocs
and
> >> >> >> > > > linking to it
> >> >> >> > from
> >> >> >> > > > our website. That should help a little, but
is obviously
> >> >> >> > > > not what he
> >> >> >> > was
> >> >> >> > > > mentioning. A proper "getting started" guide
would also
> >> >> >> > > > be great I
> >> >> >> > think.
> >> >> >> > > > I'm curious which aspects we should stress
then.
> >> >> >> > > > Especially looking at people not from Human
Inference /
> Neopost ....
> >> >> >> > > > How did you learn about MetaModel and which
questions did
> >> >> >> > > > you have (still
> >> >> >> > > > have?) and how did
> >> >> >> > you
> >> >> >> > > > get started.
> >> >> >> > > >
> >> >> >> > > > Best regards,
> >> >> >> > > > Kasper
> >> >> >> > > >
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >>
> >> >
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message