metamodel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kasper Sørensen <i.am.kasper.soren...@gmail.com>
Subject Re: [DISCUSS] Documentation improvements?
Date Wed, 21 Jan 2015 11:43:09 GMT
Figure it out :-) You have access now.

2015-01-21 12:38 GMT+01:00 Kasper Sørensen <i.am.kasper.sorensen@gmail.com>:

> I've been granted admin rights now ... trying to figure out how to give
> you access to write new pages Tomasz ... :-)
>
> 2015-01-20 16:47 GMT+01:00 Tomasz Guziałek <
> Tomasz.Guzialek@humaninference.com>:
>
>> I forgot about it actually. However, my INFRA ticket is still unresolved:
>> https://issues.apache.org/jira/browse/INFRA-8902 Hopefully, some of you
>> can add me as a contributor to the Wiki when you get your admin rights back.
>>
>> Best regards,
>> Tomasz
>>
>> -----Original Message-----
>> From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
>> Sent: Wednesday, January 14, 2015 23:22
>> To: dev@metamodel.apache.org
>> Subject: Re: [DISCUSS] Documentation improvements?
>>
>> Hi everybody,
>>
>> I know it's been a while, but I felt like bumping this topic. I was
>> thinking a bit about "what's most important for MetaModel at this point"
>> and I think documentation is high on my list, if not highest. If the wiki
>> is in any way blocking work (Tomasz?) then please raise it here and let's
>> get it fixed. And while waiting for such a thing we should still be able to
>> work on the docs - either via emails, documentation in .md files or
>> whatever.
>>
>> Cheers,
>> Kasper
>>
>> 2014-12-18 21:20 GMT+01:00 Kasper Sørensen <
>> i.am.kasper.sorensen@gmail.com>:
>>
>> > Javadocs are now available at:
>> >
>> > http://metamodel.apache.org/apidocs/3.4.1
>> >
>> > or via this redirect url that we can always make point to new versions:
>> > http://metamodel.apache.org/apidocs/current
>> >
>> > Kasper
>> >
>> >
>> > 2014-12-17 17:02 GMT+01:00 Henry Saputra <henry.saputra@gmail.com>:
>> >>
>> >> Hi Tomasz,
>> >>
>> >> Hmm somehow I also do not have admin access to add individual to edit
>> >> access, unfortunately you need to file ASF INFRA ticket to get help
>> >> on this =(
>> >>
>> >> - Henry
>> >>
>> >> On Wed, Dec 17, 2014 at 3:33 AM, Tomasz Guziałek
>> >> <Tomasz.Guzialek@humaninference.com> wrote:
>> >> > I registered myself at the Wiki, but I believe I do not have the
>> >> > rights
>> >> for editing. Henry, is that you who can fix that for me? My username
>> >> is "tomaszguzialek". Thank you!
>> >> >
>> >> > Best regards,
>> >> > Tomasz
>> >> >
>> >> > -----Original Message-----
>> >> > From: Kasper Sørensen [mailto:i.am.kasper.sorensen@gmail.com]
>> >> > Sent: Tuesday, December 16, 2014 20:04
>> >> > To: dev@metamodel.apache.org
>> >> > Subject: Re: [DISCUSS] Documentation improvements?
>> >> >
>> >> > Hi Tomasz,
>> >> >
>> >> > Good info too! I like the analogies to existing implementations - I
>> >> > can
>> >> imagine these are valueable examples for implementors.
>> >> >
>> >> > Maybe you could start a page on the MM wiki about this? To begin
>> >> > with
>> >> it could even be just a dump of that email. We can adjust and refine
>> >> it easily when it's in that wiki format I think.
>> >> >
>> >> > I would call this page something like "GettingStartedNewConnectors".
>> >> >
>> >> > Best regards,
>> >> > Kasper
>> >> >
>> >> > 2014-12-15 12:48 GMT+01:00 Tomasz Guziałek <
>> >> > Tomasz.Guzialek@humaninference.com>:
>> >> >>
>> >> >> Hello Alberto and others,
>> >> >>
>> >> >> My first thought also was around building new connectors - what
is
>> >> >> the minimum I need to provide as a developer to query a new
>> >> >> datastore and what I will get "for free" from MetaModel. My (quick
>> >> >> sketch of) step-by-step guide would look like this:
>> >> >>
>> >> >> 1) Create a new XXXDataContext class that extends
>> >> >> QueryPostProcessDataContext (to get functionalities for free) that
>> >> >> holds a reference to a native API object. Taking existing examples:
>> >> >>         CouchDbDataContext holds Ektorp' CouchDbInstance
>> >> >>         HBaseDataContext holds HTablePool
>> >> >>         CsvDataContext holds a handle to a file (through
>> >> >> MetaModel's Resource class, to be precise)
>> >> >> 2) Implement materializeMainSchemaTable to fetch the data that
is
>> >> >> going to represent the table, using the native API. Existing
>> examples:
>> >> >>         CouchDbDataContext fetches a view will all docs through
>> >> >> Ektorp's CouchDbInstance
>> >> >>         HBaseDataContext creates a Scan of a table through
>> >> HBaseTablePool
>> >> >>         CsvDataContext reads the whole file using our file handle
>> >> >> 3) Return a new XXXDataSet instance with the native result passed
>> >> >> as a parameter. XXXDataSet class takes the native result object
>> >> >> and translates it to the MetaModel's Row objects.
>> >> >>         CouchDbDataSet will parse the JSON document and
>> >> >> instantiate MetaModel's Row object with this data
>> >> >>         HBaseDataSet extracts the values from the Scan and
>> >> >> instantiates MetaModel's Row object with the values
>> >> >>         CsvDataSet translates a line from a CSV file into
>> >> >> MetaModel's Row object
>> >> >>
>> >> >> This is the minimum that needs to be implemented. While we have
>> >> >> the native result translated to MetaModel's Row objects, selecting
>> >> >> specific columns, filtering and so on we get for free from
>> >> >> QueryPostProcessDataContext class that we subclassed. Of course,
>> >> >> it is not the most performant way of querying, for example doing
a
>> >> >> full scan to get a single row by its primary key is optimized in
>> many databases.
>> >> >> Delegating primary key lookups, count queries, queries with simple
>> >> >> WHERE clauses to the native API instead of post-processing it in
>> >> >> Java is the next step for a developer of a new MetaModel connector.
>> >> >>
>> >> >> I would benefit a lot from such a guide at the beginning of my
>> >> >> journey with MetaModel...
>> >> >>
>> >> >> Best regards,
>> >> >> Tomasz
>> >> >>
>> >> >>
>> >> >> -----Original Message-----
>> >> >> From: Alberto Rodriguez [mailto:ardlema@gmail.com]
>> >> >> Sent: Friday, December 12, 2014 11:42
>> >> >> To: dev@metamodel.apache.org
>> >> >> Cc: juan.rodriguez.hortala@gmail.com
>> >> >> Subject: Re: [DISCUSS] Documentation improvements?
>> >> >>
>> >> >> Hi all,
>> >> >>
>> >> >> When I first came across the project I wasn't sure how and where
>> >> >> MM
>> >> "plays"
>> >> >> with the data. I mean, for the different datasources I didn't know
>> >> >> whether MM was fetching the data in memory and then performing
>> >> >> joins, selects and so on or was delegating this behaviour to the
>> >> >> native drivers. The table that you guys put together in the wiki:
>> >> >> QueryExecutionStrategies <
>> >> >> https://wiki.apache.org/metamodel/QueryExecutionStrategies>
is
>> >> >> quite
>> >> clarifying.
>> >> >>
>> >> >> In my particular case, as we needed new connectors that wasn't
>> >> >> implemented yet and decided to contribute to the project
>> >> >> implementing them, I was "fighting" with the code from day 0 and
I
>> >> >> must say that the code is pretty self-explanatory. Kasper has been
>> >> >> done a great job and if you have some Java experience you might
>> >> >> have a new connector ready-to-use within few days. I think we
>> >> >> should make an effort and try to share this with the world, MM
is
>> >> >> great as it is now but I think one of its main powerful features
>> >> >> is that you can create your own connectors without much effort.
It
>> >> >> might be a good idea to create a section on the website like
>> >> >> "Create your own connectors" and a guide
>> >> to implement them. What do you think?
>> >> >>
>> >> >> I would also be more than happy to help in any decision we take
>> >> >> regarding the documentation.
>> >> >>
>> >> >> Kind regards,
>> >> >>
>> >> >> 2014-12-12 9:32 GMT+01:00 Kasper Sørensen
>> >> >> <i.am.kasper.sorensen@gmail.com
>> >> >> >:
>> >> >> >
>> >> >> > Hi Juan,
>> >> >> >
>> >> >> > Thanks for that feedback. I think it helps a lot to see what
>> >> >> > relevant and good information we already have (some of it
on
>> >> >> > third party sites
>> >> >> > - we should look to integrate it some more) and also what
we
>> >> >> > should improve (the javadoc thing is such a simple thing -
I'll
>> >> >> > create a JIRA issue about doing it ASAP).
>> >> >> >
>> >> >> > Kasper
>> >> >> >
>> >> >> > 2014-12-11 20:36 GMT+01:00 Juan Rodríguez Hortalá <
>> >> >> > juan.rodriguez.hortala@gmail.com>:
>> >> >> > >
>> >> >> > > Hi all,
>> >> >> > >
>> >> >> > > I don't have a lot of experience with Metamodel, I did
my
>> >> >> > > first
>> >> >> > experiments
>> >> >> > > with it in the capstone project for a MSc, and then I
used it
>> >> >> > > in my work combined with Apache Sqoop to import data
from a
>> >> >> > > relational database,
>> >> >> > where
>> >> >> > > Metamodel was in charge of obtaining the data schema.
But
>> >> >> > > maybe my
>> >> >> > limited
>> >> >> > > experience could be of use for answering this question.
>> >> >> > > I think I discovered Metamodel when searching the internet
for
>> >> >> > > a way to connect a BI visualization tool with HBase.
As the
>> >> >> > > starting points I
>> >> >> > used:
>> >> >> > >
>> >> >> > >  * This post
>> >> >> > > http://www.infoq.com/articles/apache-incubator-metamodel
by
>> >> >> > > Kasper, which I think is a pretty good quick start, with
>> >> >> > > several
>> >> code snippets.
>> >> >> > >  * This other post
>> >> >> > > http://kasper.eobjects.org/2013/06/introducing-apache-metamode
>> >> >> > > l.ht ml by Kasper, which is more about concepts and the
origin
>> >> >> > > of Metamodel.
>> >> >> > >  * The code examples from the site
>> >> >> > > http://metamodel.apache.org/#examples
>> >> >> > > also help.
>> >> >> > >  * I also generated the Javadoc documentation, and put
it in
>> >> >> > > my github
>> >> >> > for
>> >> >> > > easy access
>> >> >> > > http://juanrh.github.io/doc/ApacheMetamodel/apidocs/
>> >> >> > > (I hope that is ok for you, by the way!).
>> >> >> > >
>> >> >> > > As I said, I haven't done anything much sophisticated,
but I
>> >> >> > > do think the javadoc documentation and API structure
is pretty
>> >> >> > > intuitive. What I miss most is documentation about the
design
>> >> >> > > and architecture so I can
>> >> >> > understand
>> >> >> > > the implications in performance of using Metamodel in
>> >> >> > > different ways. I think that is important because that
could
>> >> >> > > determine whether Metamodel could be applied or not for
a
>> >> >> > > particular problem. But these are just my personal subjective
>> impressions.
>> >> >> > >
>> >> >> > > TL;DR: In my opinion the blog post
>> >> >> > > http://www.infoq.com/articles/apache-incubator-metamodel
is a
>> >> >> > > good candidate for the "getting started" guide for Metamodel.
>> >> >> > > Also an updated version of the javadoc documentation
hosted in
>> >> >> > > the site would
>> >> >> be nice.
>> >> >> > >
>> >> >> > > Greetings,
>> >> >> > >
>> >> >> > > Juan
>> >> >> > >
>> >> >> > >
>> >> >> > > 2014-12-11 18:47 GMT+01:00 Kasper Sørensen <
>> >> >> > i.am.kasper.sorensen@gmail.com
>> >> >> > > >:
>> >> >> > >
>> >> >> > > > Hi all,
>> >> >> > > >
>> >> >> > > > Discussed with a colleague the other day the perception
of
>> >> >> > > > MetaModel
>> >> >> > when
>> >> >> > > > coming to the project as a newcomer (new in our
company and
>> >> >> > > > approaching
>> >> >> > > the
>> >> >> > > > community). He mentioned that the documentation
was too
>> >> >> > > > sparse for him
>> >> >> > to
>> >> >> > > > get a good insight by just browsing our website
and wiki. A
>> >> >> > > > proper
>> >> >> > > getting
>> >> >> > > > started and better reference documentation would
have helped
>> >> >> > > > him a lot
>> >> >> > -
>> >> >> > > > and improved his overall impression (which has improved
as
>> >> >> > > > he started
>> >> >> > > using
>> >> >> > > > it, but wasn't immediately too good).
>> >> >> > > >
>> >> >> > > > I think that's pretty good feedback - documentation
is
>> >> >> > > > important and we have not spent a lot of time on
it I guess.
>> >> >> > > >
>> >> >> > > > So are there any "low hanging fruit" we can start
by grabbing?
>> >> >> > > > And what should be the more dedicated efforts that
we go for
>> >> here?
>> >> >> > > >
>> >> >> > > > I was thinking of at least publishing our javadocs
and
>> >> >> > > > linking to it
>> >> >> > from
>> >> >> > > > our website. That should help a little, but is obviously
not
>> >> >> > > > what he
>> >> >> > was
>> >> >> > > > mentioning. A proper "getting started" guide would
also be
>> >> >> > > > great I
>> >> >> > think.
>> >> >> > > > I'm curious which aspects we should stress then.
Especially
>> >> >> > > > looking at people not from Human Inference / Neopost
....
>> >> >> > > > How did you learn about MetaModel and which questions
did
>> >> >> > > > you have (still
>> >> >> > > > have?) and how did
>> >> >> > you
>> >> >> > > > get started.
>> >> >> > > >
>> >> >> > > > Best regards,
>> >> >> > > > Kasper
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message