metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Saputra <henry.sapu...@gmail.com>
Subject Re: [DISCUSS] Documentation improvements?
Date Wed, 17 Dec 2014 16:02:55 GMT
That is weird, let me play around with it again

On Wed, Dec 17, 2014 at 6:20 AM, Tomasz Guziałek
<Tomasz.Guzialek@humaninference.com> wrote:
> Hello Alberto,
>
> I have found your INFRA JIRA issue: https://issues.apache.org/jira/browse/INFRA-8508
The comment suggests that Matt Franklin and Henry Saputra as MetaModel's Wiki admins have
rights to grant the access (so INFRA ticket is kind of mistargetted). However, I also found
an older post in the mailing list where Henry suggests creating one, so I am a little confused
to be honest. Henry, could you clarify that?
>
> Best regards,
> Tomasz
>
> -----Original Message-----
> From: Alberto Rodriguez [mailto:ardlema@gmail.com]
> Sent: Wednesday, December 17, 2014 14:48
> To: dev@metamodel.apache.org
> Subject: Re: [DISCUSS] Documentation improvements?
>
> Hi Tomasz,
>
> You have to open a ticket in the infra project's jira.
>
> Regards,
>
> Alberto
>
> 2014-12-17 12:33 GMT+01:00 Tomasz Guziałek <
> Tomasz.Guzialek@humaninference.com>:
>>
>> I registered myself at the Wiki, but I believe I do not have the
>> rights for editing. Henry, is that you who can fix that for me? My
>> username is "tomaszguzialek". Thank you!
>>
>> Best regards,
>> Tomasz
>>
>> -----Original Message-----
>> From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
>> Sent: Tuesday, December 16, 2014 20:04
>> To: dev@metamodel.apache.org
>> Subject: Re: [DISCUSS] Documentation improvements?
>>
>> Hi Tomasz,
>>
>> Good info too! I like the analogies to existing implementations - I
>> can imagine these are valueable examples for implementors.
>>
>> Maybe you could start a page on the MM wiki about this? To begin with
>> it could even be just a dump of that email. We can adjust and refine
>> it easily when it's in that wiki format I think.
>>
>> I would call this page something like "GettingStartedNewConnectors".
>>
>> Best regards,
>> Kasper
>>
>> 2014-12-15 12:48 GMT+01:00 Tomasz Guziałek <
>> Tomasz.Guzialek@humaninference.com>:
>> >
>> > Hello Alberto and others,
>> >
>> > My first thought also was around building new connectors - what is
>> > the minimum I need to provide as a developer to query a new
>> > datastore and what I will get "for free" from MetaModel. My (quick
>> > sketch of) step-by-step guide would look like this:
>> >
>> > 1) Create a new XXXDataContext class that extends
>> > QueryPostProcessDataContext (to get functionalities for free) that
>> > holds a reference to a native API object. Taking existing examples:
>> >         CouchDbDataContext holds Ektorp' CouchDbInstance
>> >         HBaseDataContext holds HTablePool
>> >         CsvDataContext holds a handle to a file (through MetaModel's
>> > Resource class, to be precise)
>> > 2) Implement materializeMainSchemaTable to fetch the data that is
>> > going to represent the table, using the native API. Existing examples:
>> >         CouchDbDataContext fetches a view will all docs through
>> > Ektorp's CouchDbInstance
>> >         HBaseDataContext creates a Scan of a table through HBaseTablePool
>> >         CsvDataContext reads the whole file using our file handle
>> > 3) Return a new XXXDataSet instance with the native result passed as
>> > a parameter. XXXDataSet class takes the native result object and
>> > translates it to the MetaModel's Row objects.
>> >         CouchDbDataSet will parse the JSON document and instantiate
>> > MetaModel's Row object with this data
>> >         HBaseDataSet extracts the values from the Scan and
>> > instantiates MetaModel's Row object with the values
>> >         CsvDataSet translates a line from a CSV file into
>> > MetaModel's Row object
>> >
>> > This is the minimum that needs to be implemented. While we have the
>> > native result translated to MetaModel's Row objects, selecting
>> > specific columns, filtering and so on we get for free from
>> > QueryPostProcessDataContext class that we subclassed. Of course, it
>> > is not the most performant way of querying, for example doing a full
>> > scan to get a single row by its primary key is optimized in many databases.
>> > Delegating primary key lookups, count queries, queries with simple
>> > WHERE clauses to the native API instead of post-processing it in
>> > Java is the next step for a developer of a new MetaModel connector.
>> >
>> > I would benefit a lot from such a guide at the beginning of my
>> > journey with MetaModel...
>> >
>> > Best regards,
>> > Tomasz
>> >
>> >
>> > -----Original Message-----
>> > From: Alberto Rodriguez [mailto:ardlema@gmail.com]
>> > Sent: Friday, December 12, 2014 11:42
>> > To: dev@metamodel.apache.org
>> > Cc: juan.rodriguez.hortala@gmail.com
>> > Subject: Re: [DISCUSS] Documentation improvements?
>> >
>> > Hi all,
>> >
>> > When I first came across the project I wasn't sure how and where MM
>> "plays"
>> > with the data. I mean, for the different datasources I didn't know
>> > whether MM was fetching the data in memory and then performing
>> > joins, selects and so on or was delegating this behaviour to the
>> > native drivers. The table that you guys put together in the wiki:
>> > QueryExecutionStrategies <
>> > https://wiki.apache.org/metamodel/QueryExecutionStrategies> is quite
>> clarifying.
>> >
>> > In my particular case, as we needed new connectors that wasn't
>> > implemented yet and decided to contribute to the project
>> > implementing them, I was "fighting" with the code from day 0 and I
>> > must say that the code is pretty self-explanatory. Kasper has been
>> > done a great job and if you have some Java experience you might have
>> > a new connector ready-to-use within few days. I think we should make
>> > an effort and try to share this with the world, MM is great as it is
>> > now but I think one of its main powerful features is that you can
>> > create your own connectors without much effort. It might be a good
>> > idea to create a section on the website like "Create your own
>> > connectors" and a guide to
>> implement them. What do you think?
>> >
>> > I would also be more than happy to help in any decision we take
>> > regarding the documentation.
>> >
>> > Kind regards,
>> >
>> > 2014-12-12 9:32 GMT+01:00 Kasper Sørensen
>> > <i.am.kasper.sorensen@gmail.com
>> > >:
>> > >
>> > > Hi Juan,
>> > >
>> > > Thanks for that feedback. I think it helps a lot to see what
>> > > relevant and good information we already have (some of it on third
>> > > party sites
>> > > - we should look to integrate it some more) and also what we
>> > > should improve (the javadoc thing is such a simple thing - I'll
>> > > create a JIRA issue about doing it ASAP).
>> > >
>> > > Kasper
>> > >
>> > > 2014-12-11 20:36 GMT+01:00 Juan Rodríguez Hortalá <
>> > > juan.rodriguez.hortala@gmail.com>:
>> > > >
>> > > > Hi all,
>> > > >
>> > > > I don't have a lot of experience with Metamodel, I did my first
>> > > experiments
>> > > > with it in the capstone project for a MSc, and then I used it in
>> > > > my work combined with Apache Sqoop to import data from a
>> > > > relational database,
>> > > where
>> > > > Metamodel was in charge of obtaining the data schema. But maybe
>> > > > my
>> > > limited
>> > > > experience could be of use for answering this question.
>> > > > I think I discovered Metamodel when searching the internet for a
>> > > > way to connect a BI visualization tool with HBase. As the
>> > > > starting points I
>> > > used:
>> > > >
>> > > >  * This post
>> > > > http://www.infoq.com/articles/apache-incubator-metamodel by
>> > > > Kasper, which I think is a pretty good quick start, with several
>> code snippets.
>> > > >  * This other post
>> > > > http://kasper.eobjects.org/2013/06/introducing-apache-metamodel.
>> > > > ht ml by Kasper, which is more about concepts and the origin of
>> > > > Metamodel.
>> > > >  * The code examples from the site
>> > > > http://metamodel.apache.org/#examples
>> > > > also help.
>> > > >  * I also generated the Javadoc documentation, and put it in my
>> > > > github
>> > > for
>> > > > easy access http://juanrh.github.io/doc/ApacheMetamodel/apidocs/
>> > > > (I hope that is ok for you, by the way!).
>> > > >
>> > > > As I said, I haven't done anything much sophisticated, but I do
>> > > > think the javadoc documentation and API structure is pretty
>> > > > intuitive. What I miss most is documentation about the design
>> > > > and architecture so I can
>> > > understand
>> > > > the implications in performance of using Metamodel in different
>> > > > ways. I think that is important because that could determine
>> > > > whether Metamodel could be applied or not for a particular
>> > > > problem. But these are just my personal subjective impressions.
>> > > >
>> > > > TL;DR: In my opinion the blog post
>> > > > http://www.infoq.com/articles/apache-incubator-metamodel is a
>> > > > good candidate for the "getting started" guide for Metamodel.
>> > > > Also an updated version of the javadoc documentation hosted in
>> > > > the site would
>> > be nice.
>> > > >
>> > > > Greetings,
>> > > >
>> > > > Juan
>> > > >
>> > > >
>> > > > 2014-12-11 18:47 GMT+01:00 Kasper Sørensen <
>> > > i.am.kasper.sorensen@gmail.com
>> > > > >:
>> > > >
>> > > > > Hi all,
>> > > > >
>> > > > > Discussed with a colleague the other day the perception of
>> > > > > MetaModel
>> > > when
>> > > > > coming to the project as a newcomer (new in our company and
>> > > > > approaching
>> > > > the
>> > > > > community). He mentioned that the documentation was too sparse
>> > > > > for him
>> > > to
>> > > > > get a good insight by just browsing our website and wiki. A
>> > > > > proper
>> > > > getting
>> > > > > started and better reference documentation would have helped
>> > > > > him a lot
>> > > -
>> > > > > and improved his overall impression (which has improved as he
>> > > > > started
>> > > > using
>> > > > > it, but wasn't immediately too good).
>> > > > >
>> > > > > I think that's pretty good feedback - documentation is
>> > > > > important and we have not spent a lot of time on it I guess.
>> > > > >
>> > > > > So are there any "low hanging fruit" we can start by grabbing?
>> > > > > And what should be the more dedicated efforts that we go for
here?
>> > > > >
>> > > > > I was thinking of at least publishing our javadocs and linking
>> > > > > to it
>> > > from
>> > > > > our website. That should help a little, but is obviously not
>> > > > > what he
>> > > was
>> > > > > mentioning. A proper "getting started" guide would also be
>> > > > > great I
>> > > think.
>> > > > > I'm curious which aspects we should stress then. Especially
>> > > > > looking at people not from Human Inference / Neopost .... How
>> > > > > did you learn about MetaModel and which questions did you have
>> > > > > (still
>> > > > > have?) and how did
>> > > you
>> > > > > get started.
>> > > > >
>> > > > > Best regards,
>> > > > > Kasper
>> > > > >
>> > > >
>> > >
>> >
>>

Mime
View raw message