airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saminda Wijeratne <samin...@gmail.com>
Subject Re: Airavata Thrift Module Hierarchy
Date Mon, 10 Feb 2014 01:32:38 GMT
On Sat, Feb 8, 2014 at 11:05 AM, Suresh Marru <smarru@apache.org> wrote:

> On Feb 8, 2014, at 1:06 PM, Amila Jayasekara <thejaka.amila@gmail.com>
> wrote:
>
> > On Fri, Feb 7, 2014 at 8:49 PM, Suresh Marru <smarru@apache.org> wrote:
> > On Feb 7, 2014, at 2:59 PM, Saminda Wijeratne <samindaw@gmail.com>
> wrote:
> >
> > > I thought all client SDK sources gets auto-generated from the thrift
> file. Am I missing something here?
> >
> > They are generated but not auto-generated. The default thrift generator
> is written in c++ which becomes platform specific. So we need to instal
> thrift and generate and put it into the client SDK packages. There are
> third party auto-generators like this from twitter -
> http://twitter.github.io/scrooge/ I think for now we can live with manual
> generation and everything else stabilizes we can explore these options.
> >
> > Hi Suresh,
> >
> > Could you explain what you mean by manual generation ?
>
> I mean the generated sources are not auto-generated at build time like for
> instance from XSD's we generate the xmlbeans as a maven task. There is a
> thrift code generation maven plugin [1] which is now being integrated into
> next thrift release, but it will also require local thrift installation. So
> we cannot enable by default.
>
> > Isnt it possible to automate code generation at least using a shell
> script ?
>
> Yes, I am close to finishing the script [2], it works as of now, but I am
> planning to change the checksome check from 'cmp' to use rsync recursive
> compare and sync changed files. Other than that, the scripts works for now.
>
> > Downside of manual generation is it could easily lead to outdated
> generated code (i.e. thrift files and generated code does not match).
>
> This is a big risk, unfortunately we have to rely on developer discipline
> to ensure this. We need to clearly document that any changes to thrift
> files should be followed by running the generate-thrift-files.sh script and
> checkin the updated generated code along with modified thrift files.
> Hopefully we have to very minimally change the thrift files and be vigilant
> on watching the thrift file commits.
>
Until we do find a permanent solution the integration tests would take care
of making sure the thrift API is up to date. But we need to expand the
integration tests to cover all functions in the API in that case.

Saminda

>
> Suresh
>
> [1] - https://github.com/dtrott/maven-thrift-plugin
> [2] -
> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob_plain;f=airavata-api/generate-thrift-files.sh;hb=299872ab6cb35d25e16d62c49940b2799df29b05
>
> >
> > Thanks
> > Amila
> >
> > For now I am foreseeing we distribute the client sdk's similar to
> evernote distribution:
> > http://dev.evernote.com/doc/
> > https://github.com/evernote
> >
> > Suresh
> >
> >
> > >
> > >
> > > On Fri, Feb 7, 2014 at 10:27 AM, Suresh Marru <smarru@apache.org>
> wrote:
> > > Hi Saminda,
> > >
> > > As with other distributions, the client SDK's will have sources and we
> can ship the binaries as release artifacts. Ofcourse the java client sdk
> can be pushed to maven. It will be nice to push the PHP SDK to PEAR library
> (http://pear.php.net/), perl to CPAN (http://www.cpan.org/) but I do not
> know whats involved and associate release tasks. But it will be good to
> explore once we get some php users.
> > >
> > > To start with, I suggested to move the airavata-api, the data models
> and client sdk's to top level in the repository. So now users who are
> interested in checking the source of data model and clients can do so right
> away. The server skeletons and handlers are also here. Only core developers
> who work with component implementations can navigate into modules.
> > >
> > > Suresh
> > >
> > > On Feb 6, 2014, at 4:12 PM, Saminda Wijeratne <samindaw@gmail.com>
> wrote:
> > >
> > > > I'm sorry I forgot to ask this question earlier (and probably it was
> asked earlier in a discussion which I missed following). The client SDKs
> are straightaway binary distributions (zip/tar.gz)?
> > > >
> > > >
> > > > On Thu, Feb 6, 2014 at 8:10 AM, Suresh Marru <smarru@apache.org>
> wrote:
> > > > With no objections, I will goahead and make this directory structure
> and start checking in thrift generated code.
> > > >
> > > > Suresh
> > > > On Feb 4, 2014, at 12:42 PM, Suresh Marru <smarru@apache.org> wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > As we are discussing on [1] Of all the examples I looked around,
I
> really liked apache accumulo approach in working with thrift (thanks for
> the pointer Supun). How about we shamelessly copy clone accumulo's script?
> It nicely appends license headers and cherry picks thrift interfaces
> changes and copies them into the right destination.
> > > > >
> > > > > For the source organization, how about the following:
> > > > >
> > > > > Airavata-Root
> > > > >  |- modules
> > > > >  |   |- ....
> > > > >  |   |- ....
> > > > >  |- airavata-api
> > > > >  |   |-pom.xml
> > > > >  |   |-generate-thrift-files.sh
> > > > >  |   |-thrift-idls
> > > > >  |   |-datamodel
> > > > >  |   |-server
> > > > >  |   |-server-handlers
> > > > >  |   |-client-sdks
> > > > >  |   |   |-java
> > > > >  |   |   |-php
> > > > >  |   |   |-js
> > > > >  |   |   |-.....
> > > > >
> > > > > Suresh
> > > > > [1] - http://markmail.org/message/f6bxanaxvke7boer
> > > >
> > > >
> > >
> > >
> >
> >
>
>

Mime
View raw message