metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomasz Guziałek <Tomasz.Guzia...@HumanInference.com>
Subject RE: [DISCUSS] Documentation improvements?
Date Tue, 20 Jan 2015 15:47:45 GMT
I forgot about it actually. However, my INFRA ticket is still unresolved: https://issues.apache.org/jira/browse/INFRA-8902
Hopefully, some of you can add me as a contributor to the Wiki when you get your admin rights
back.

Best regards,
Tomasz

-----Original Message-----
From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com] 
Sent: Wednesday, January 14, 2015 23:22
To: dev@metamodel.apache.org
Subject: Re: [DISCUSS] Documentation improvements?

Hi everybody,

I know it's been a while, but I felt like bumping this topic. I was thinking a bit about "what's
most important for MetaModel at this point"
and I think documentation is high on my list, if not highest. If the wiki is in any way blocking
work (Tomasz?) then please raise it here and let's get it fixed. And while waiting for such
a thing we should still be able to work on the docs - either via emails, documentation in
.md files or whatever.

Cheers,
Kasper

2014-12-18 21:20 GMT+01:00 Kasper Sørensen <i.am.kasper.sorensen@gmail.com>:

> Javadocs are now available at:
>
> http://metamodel.apache.org/apidocs/3.4.1
>
> or via this redirect url that we can always make point to new versions:
> http://metamodel.apache.org/apidocs/current
>
> Kasper
>
>
> 2014-12-17 17:02 GMT+01:00 Henry Saputra <henry.saputra@gmail.com>:
>>
>> Hi Tomasz,
>>
>> Hmm somehow I also do not have admin access to add individual to edit 
>> access, unfortunately you need to file ASF INFRA ticket to get help 
>> on this =(
>>
>> - Henry
>>
>> On Wed, Dec 17, 2014 at 3:33 AM, Tomasz Guziałek 
>> <Tomasz.Guzialek@humaninference.com> wrote:
>> > I registered myself at the Wiki, but I believe I do not have the 
>> > rights
>> for editing. Henry, is that you who can fix that for me? My username 
>> is "tomaszguzialek". Thank you!
>> >
>> > Best regards,
>> > Tomasz
>> >
>> > -----Original Message-----
>> > From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
>> > Sent: Tuesday, December 16, 2014 20:04
>> > To: dev@metamodel.apache.org
>> > Subject: Re: [DISCUSS] Documentation improvements?
>> >
>> > Hi Tomasz,
>> >
>> > Good info too! I like the analogies to existing implementations - I 
>> > can
>> imagine these are valueable examples for implementors.
>> >
>> > Maybe you could start a page on the MM wiki about this? To begin 
>> > with
>> it could even be just a dump of that email. We can adjust and refine 
>> it easily when it's in that wiki format I think.
>> >
>> > I would call this page something like "GettingStartedNewConnectors".
>> >
>> > Best regards,
>> > Kasper
>> >
>> > 2014-12-15 12:48 GMT+01:00 Tomasz Guziałek <
>> > Tomasz.Guzialek@humaninference.com>:
>> >>
>> >> Hello Alberto and others,
>> >>
>> >> My first thought also was around building new connectors - what is 
>> >> the minimum I need to provide as a developer to query a new 
>> >> datastore and what I will get "for free" from MetaModel. My (quick 
>> >> sketch of) step-by-step guide would look like this:
>> >>
>> >> 1) Create a new XXXDataContext class that extends 
>> >> QueryPostProcessDataContext (to get functionalities for free) that 
>> >> holds a reference to a native API object. Taking existing examples:
>> >>         CouchDbDataContext holds Ektorp' CouchDbInstance
>> >>         HBaseDataContext holds HTablePool
>> >>         CsvDataContext holds a handle to a file (through 
>> >> MetaModel's Resource class, to be precise)
>> >> 2) Implement materializeMainSchemaTable to fetch the data that is 
>> >> going to represent the table, using the native API. Existing examples:
>> >>         CouchDbDataContext fetches a view will all docs through 
>> >> Ektorp's CouchDbInstance
>> >>         HBaseDataContext creates a Scan of a table through
>> HBaseTablePool
>> >>         CsvDataContext reads the whole file using our file handle
>> >> 3) Return a new XXXDataSet instance with the native result passed 
>> >> as a parameter. XXXDataSet class takes the native result object 
>> >> and translates it to the MetaModel's Row objects.
>> >>         CouchDbDataSet will parse the JSON document and 
>> >> instantiate MetaModel's Row object with this data
>> >>         HBaseDataSet extracts the values from the Scan and 
>> >> instantiates MetaModel's Row object with the values
>> >>         CsvDataSet translates a line from a CSV file into 
>> >> MetaModel's Row object
>> >>
>> >> This is the minimum that needs to be implemented. While we have 
>> >> the native result translated to MetaModel's Row objects, selecting 
>> >> specific columns, filtering and so on we get for free from 
>> >> QueryPostProcessDataContext class that we subclassed. Of course, 
>> >> it is not the most performant way of querying, for example doing a 
>> >> full scan to get a single row by its primary key is optimized in many databases.
>> >> Delegating primary key lookups, count queries, queries with simple 
>> >> WHERE clauses to the native API instead of post-processing it in 
>> >> Java is the next step for a developer of a new MetaModel connector.
>> >>
>> >> I would benefit a lot from such a guide at the beginning of my 
>> >> journey with MetaModel...
>> >>
>> >> Best regards,
>> >> Tomasz
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Alberto Rodriguez [mailto:ardlema@gmail.com]
>> >> Sent: Friday, December 12, 2014 11:42
>> >> To: dev@metamodel.apache.org
>> >> Cc: juan.rodriguez.hortala@gmail.com
>> >> Subject: Re: [DISCUSS] Documentation improvements?
>> >>
>> >> Hi all,
>> >>
>> >> When I first came across the project I wasn't sure how and where 
>> >> MM
>> "plays"
>> >> with the data. I mean, for the different datasources I didn't know 
>> >> whether MM was fetching the data in memory and then performing 
>> >> joins, selects and so on or was delegating this behaviour to the 
>> >> native drivers. The table that you guys put together in the wiki:
>> >> QueryExecutionStrategies <
>> >> https://wiki.apache.org/metamodel/QueryExecutionStrategies> is 
>> >> quite
>> clarifying.
>> >>
>> >> In my particular case, as we needed new connectors that wasn't 
>> >> implemented yet and decided to contribute to the project 
>> >> implementing them, I was "fighting" with the code from day 0 and I 
>> >> must say that the code is pretty self-explanatory. Kasper has been 
>> >> done a great job and if you have some Java experience you might 
>> >> have a new connector ready-to-use within few days. I think we 
>> >> should make an effort and try to share this with the world, MM is 
>> >> great as it is now but I think one of its main powerful features 
>> >> is that you can create your own connectors without much effort. It 
>> >> might be a good idea to create a section on the website like 
>> >> "Create your own connectors" and a guide
>> to implement them. What do you think?
>> >>
>> >> I would also be more than happy to help in any decision we take 
>> >> regarding the documentation.
>> >>
>> >> Kind regards,
>> >>
>> >> 2014-12-12 9:32 GMT+01:00 Kasper Sørensen 
>> >> <i.am.kasper.sorensen@gmail.com
>> >> >:
>> >> >
>> >> > Hi Juan,
>> >> >
>> >> > Thanks for that feedback. I think it helps a lot to see what 
>> >> > relevant and good information we already have (some of it on 
>> >> > third party sites
>> >> > - we should look to integrate it some more) and also what we 
>> >> > should improve (the javadoc thing is such a simple thing - I'll 
>> >> > create a JIRA issue about doing it ASAP).
>> >> >
>> >> > Kasper
>> >> >
>> >> > 2014-12-11 20:36 GMT+01:00 Juan Rodríguez Hortalá <
>> >> > juan.rodriguez.hortala@gmail.com>:
>> >> > >
>> >> > > Hi all,
>> >> > >
>> >> > > I don't have a lot of experience with Metamodel, I did my 
>> >> > > first
>> >> > experiments
>> >> > > with it in the capstone project for a MSc, and then I used it

>> >> > > in my work combined with Apache Sqoop to import data from a 
>> >> > > relational database,
>> >> > where
>> >> > > Metamodel was in charge of obtaining the data schema. But 
>> >> > > maybe my
>> >> > limited
>> >> > > experience could be of use for answering this question.
>> >> > > I think I discovered Metamodel when searching the internet for

>> >> > > a way to connect a BI visualization tool with HBase. As the 
>> >> > > starting points I
>> >> > used:
>> >> > >
>> >> > >  * This post
>> >> > > http://www.infoq.com/articles/apache-incubator-metamodel by 
>> >> > > Kasper, which I think is a pretty good quick start, with 
>> >> > > several
>> code snippets.
>> >> > >  * This other post
>> >> > > http://kasper.eobjects.org/2013/06/introducing-apache-metamode
>> >> > > l.ht ml by Kasper, which is more about concepts and the origin

>> >> > > of Metamodel.
>> >> > >  * The code examples from the site 
>> >> > > http://metamodel.apache.org/#examples
>> >> > > also help.
>> >> > >  * I also generated the Javadoc documentation, and put it in 
>> >> > > my github
>> >> > for
>> >> > > easy access 
>> >> > > http://juanrh.github.io/doc/ApacheMetamodel/apidocs/
>> >> > > (I hope that is ok for you, by the way!).
>> >> > >
>> >> > > As I said, I haven't done anything much sophisticated, but I 
>> >> > > do think the javadoc documentation and API structure is pretty

>> >> > > intuitive. What I miss most is documentation about the design

>> >> > > and architecture so I can
>> >> > understand
>> >> > > the implications in performance of using Metamodel in 
>> >> > > different ways. I think that is important because that could 
>> >> > > determine whether Metamodel could be applied or not for a 
>> >> > > particular problem. But these are just my personal subjective
impressions.
>> >> > >
>> >> > > TL;DR: In my opinion the blog post 
>> >> > > http://www.infoq.com/articles/apache-incubator-metamodel is a

>> >> > > good candidate for the "getting started" guide for Metamodel.

>> >> > > Also an updated version of the javadoc documentation hosted in

>> >> > > the site would
>> >> be nice.
>> >> > >
>> >> > > Greetings,
>> >> > >
>> >> > > Juan
>> >> > >
>> >> > >
>> >> > > 2014-12-11 18:47 GMT+01:00 Kasper Sørensen <
>> >> > i.am.kasper.sorensen@gmail.com
>> >> > > >:
>> >> > >
>> >> > > > Hi all,
>> >> > > >
>> >> > > > Discussed with a colleague the other day the perception of

>> >> > > > MetaModel
>> >> > when
>> >> > > > coming to the project as a newcomer (new in our company and

>> >> > > > approaching
>> >> > > the
>> >> > > > community). He mentioned that the documentation was too 
>> >> > > > sparse for him
>> >> > to
>> >> > > > get a good insight by just browsing our website and wiki.
A 
>> >> > > > proper
>> >> > > getting
>> >> > > > started and better reference documentation would have helped

>> >> > > > him a lot
>> >> > -
>> >> > > > and improved his overall impression (which has improved as

>> >> > > > he started
>> >> > > using
>> >> > > > it, but wasn't immediately too good).
>> >> > > >
>> >> > > > I think that's pretty good feedback - documentation is 
>> >> > > > important and we have not spent a lot of time on it I guess.
>> >> > > >
>> >> > > > So are there any "low hanging fruit" we can start by grabbing?
>> >> > > > And what should be the more dedicated efforts that we go
for
>> here?
>> >> > > >
>> >> > > > I was thinking of at least publishing our javadocs and 
>> >> > > > linking to it
>> >> > from
>> >> > > > our website. That should help a little, but is obviously
not 
>> >> > > > what he
>> >> > was
>> >> > > > mentioning. A proper "getting started" guide would also be

>> >> > > > great I
>> >> > think.
>> >> > > > I'm curious which aspects we should stress then. Especially

>> >> > > > looking at people not from Human Inference / Neopost ....

>> >> > > > How did you learn about MetaModel and which questions did

>> >> > > > you have (still
>> >> > > > have?) and how did
>> >> > you
>> >> > > > get started.
>> >> > > >
>> >> > > > Best regards,
>> >> > > > Kasper
>> >> > > >
>> >> > >
>> >> >
>> >>
>>
>
Mime
View raw message