metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alberto Rodriguez <ardl...@gmail.com>
Subject Re: [DISCUSS] Documentation improvements?
Date Wed, 17 Dec 2014 13:48:17 GMT
Hi Tomasz,

You have to open a ticket in the infra project's jira.

Regards,

Alberto

2014-12-17 12:33 GMT+01:00 Tomasz Guziałek <
Tomasz.Guzialek@humaninference.com>:
>
> I registered myself at the Wiki, but I believe I do not have the rights
> for editing. Henry, is that you who can fix that for me? My username is
> "tomaszguzialek". Thank you!
>
> Best regards,
> Tomasz
>
> -----Original Message-----
> From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
> Sent: Tuesday, December 16, 2014 20:04
> To: dev@metamodel.apache.org
> Subject: Re: [DISCUSS] Documentation improvements?
>
> Hi Tomasz,
>
> Good info too! I like the analogies to existing implementations - I can
> imagine these are valueable examples for implementors.
>
> Maybe you could start a page on the MM wiki about this? To begin with it
> could even be just a dump of that email. We can adjust and refine it easily
> when it's in that wiki format I think.
>
> I would call this page something like "GettingStartedNewConnectors".
>
> Best regards,
> Kasper
>
> 2014-12-15 12:48 GMT+01:00 Tomasz Guziałek <
> Tomasz.Guzialek@humaninference.com>:
> >
> > Hello Alberto and others,
> >
> > My first thought also was around building new connectors - what is the
> > minimum I need to provide as a developer to query a new datastore and
> > what I will get "for free" from MetaModel. My (quick sketch of)
> > step-by-step guide would look like this:
> >
> > 1) Create a new XXXDataContext class that extends
> > QueryPostProcessDataContext (to get functionalities for free) that
> > holds a reference to a native API object. Taking existing examples:
> >         CouchDbDataContext holds Ektorp' CouchDbInstance
> >         HBaseDataContext holds HTablePool
> >         CsvDataContext holds a handle to a file (through MetaModel's
> > Resource class, to be precise)
> > 2) Implement materializeMainSchemaTable to fetch the data that is
> > going to represent the table, using the native API. Existing examples:
> >         CouchDbDataContext fetches a view will all docs through
> > Ektorp's CouchDbInstance
> >         HBaseDataContext creates a Scan of a table through HBaseTablePool
> >         CsvDataContext reads the whole file using our file handle
> > 3) Return a new XXXDataSet instance with the native result passed as a
> > parameter. XXXDataSet class takes the native result object and
> > translates it to the MetaModel's Row objects.
> >         CouchDbDataSet will parse the JSON document and instantiate
> > MetaModel's Row object with this data
> >         HBaseDataSet extracts the values from the Scan and
> > instantiates MetaModel's Row object with the values
> >         CsvDataSet translates a line from a CSV file into MetaModel's
> > Row object
> >
> > This is the minimum that needs to be implemented. While we have the
> > native result translated to MetaModel's Row objects, selecting
> > specific columns, filtering and so on we get for free from
> > QueryPostProcessDataContext class that we subclassed. Of course, it is
> > not the most performant way of querying, for example doing a full scan
> > to get a single row by its primary key is optimized in many databases.
> > Delegating primary key lookups, count queries, queries with simple
> > WHERE clauses to the native API instead of post-processing it in Java
> > is the next step for a developer of a new MetaModel connector.
> >
> > I would benefit a lot from such a guide at the beginning of my journey
> > with MetaModel...
> >
> > Best regards,
> > Tomasz
> >
> >
> > -----Original Message-----
> > From: Alberto Rodriguez [mailto:ardlema@gmail.com]
> > Sent: Friday, December 12, 2014 11:42
> > To: dev@metamodel.apache.org
> > Cc: juan.rodriguez.hortala@gmail.com
> > Subject: Re: [DISCUSS] Documentation improvements?
> >
> > Hi all,
> >
> > When I first came across the project I wasn't sure how and where MM
> "plays"
> > with the data. I mean, for the different datasources I didn't know
> > whether MM was fetching the data in memory and then performing joins,
> > selects and so on or was delegating this behaviour to the native
> > drivers. The table that you guys put together in the wiki:
> > QueryExecutionStrategies <
> > https://wiki.apache.org/metamodel/QueryExecutionStrategies> is quite
> clarifying.
> >
> > In my particular case, as we needed new connectors that wasn't
> > implemented yet and decided to contribute to the project implementing
> > them, I was "fighting" with the code from day 0 and I must say that
> > the code is pretty self-explanatory. Kasper has been done a great job
> > and if you have some Java experience you might have a new connector
> > ready-to-use within few days. I think we should make an effort and try
> > to share this with the world, MM is great as it is now but I think one
> > of its main powerful features is that you can create your own
> > connectors without much effort. It might be a good idea to create a
> > section on the website like "Create your own connectors" and a guide to
> implement them. What do you think?
> >
> > I would also be more than happy to help in any decision we take
> > regarding the documentation.
> >
> > Kind regards,
> >
> > 2014-12-12 9:32 GMT+01:00 Kasper Sørensen
> > <i.am.kasper.sorensen@gmail.com
> > >:
> > >
> > > Hi Juan,
> > >
> > > Thanks for that feedback. I think it helps a lot to see what
> > > relevant and good information we already have (some of it on third
> > > party sites
> > > - we should look to integrate it some more) and also what we should
> > > improve (the javadoc thing is such a simple thing - I'll create a
> > > JIRA issue about doing it ASAP).
> > >
> > > Kasper
> > >
> > > 2014-12-11 20:36 GMT+01:00 Juan Rodríguez Hortalá <
> > > juan.rodriguez.hortala@gmail.com>:
> > > >
> > > > Hi all,
> > > >
> > > > I don't have a lot of experience with Metamodel, I did my first
> > > experiments
> > > > with it in the capstone project for a MSc, and then I used it in
> > > > my work combined with Apache Sqoop to import data from a
> > > > relational database,
> > > where
> > > > Metamodel was in charge of obtaining the data schema. But maybe my
> > > limited
> > > > experience could be of use for answering this question.
> > > > I think I discovered Metamodel when searching the internet for a
> > > > way to connect a BI visualization tool with HBase. As the starting
> > > > points I
> > > used:
> > > >
> > > >  * This post
> > > > http://www.infoq.com/articles/apache-incubator-metamodel by
> > > > Kasper, which I think is a pretty good quick start, with several
> code snippets.
> > > >  * This other post
> > > > http://kasper.eobjects.org/2013/06/introducing-apache-metamodel.ht
> > > > ml by Kasper, which is more about concepts and the origin of
> > > > Metamodel.
> > > >  * The code examples from the site
> > > > http://metamodel.apache.org/#examples
> > > > also help.
> > > >  * I also generated the Javadoc documentation, and put it in my
> > > > github
> > > for
> > > > easy access http://juanrh.github.io/doc/ApacheMetamodel/apidocs/
> > > > (I hope that is ok for you, by the way!).
> > > >
> > > > As I said, I haven't done anything much sophisticated, but I do
> > > > think the javadoc documentation and API structure is pretty
> > > > intuitive. What I miss most is documentation about the design and
> > > > architecture so I can
> > > understand
> > > > the implications in performance of using Metamodel in different
> > > > ways. I think that is important because that could determine
> > > > whether Metamodel could be applied or not for a particular
> > > > problem. But these are just my personal subjective impressions.
> > > >
> > > > TL;DR: In my opinion the blog post
> > > > http://www.infoq.com/articles/apache-incubator-metamodel is a good
> > > > candidate for the "getting started" guide for Metamodel. Also an
> > > > updated version of the javadoc documentation hosted in the site
> > > > would
> > be nice.
> > > >
> > > > Greetings,
> > > >
> > > > Juan
> > > >
> > > >
> > > > 2014-12-11 18:47 GMT+01:00 Kasper Sørensen <
> > > i.am.kasper.sorensen@gmail.com
> > > > >:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Discussed with a colleague the other day the perception of
> > > > > MetaModel
> > > when
> > > > > coming to the project as a newcomer (new in our company and
> > > > > approaching
> > > > the
> > > > > community). He mentioned that the documentation was too sparse
> > > > > for him
> > > to
> > > > > get a good insight by just browsing our website and wiki. A
> > > > > proper
> > > > getting
> > > > > started and better reference documentation would have helped him
> > > > > a lot
> > > -
> > > > > and improved his overall impression (which has improved as he
> > > > > started
> > > > using
> > > > > it, but wasn't immediately too good).
> > > > >
> > > > > I think that's pretty good feedback - documentation is important
> > > > > and we have not spent a lot of time on it I guess.
> > > > >
> > > > > So are there any "low hanging fruit" we can start by grabbing?
> > > > > And what should be the more dedicated efforts that we go for here?
> > > > >
> > > > > I was thinking of at least publishing our javadocs and linking
> > > > > to it
> > > from
> > > > > our website. That should help a little, but is obviously not
> > > > > what he
> > > was
> > > > > mentioning. A proper "getting started" guide would also be great
> > > > > I
> > > think.
> > > > > I'm curious which aspects we should stress then. Especially
> > > > > looking at people not from Human Inference / Neopost .... How
> > > > > did you learn about MetaModel and which questions did you have
> > > > > (still
> > > > > have?) and how did
> > > you
> > > > > get started.
> > > > >
> > > > > Best regards,
> > > > > Kasper
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message